--- title: Analysis description: Ask your eval history questions in SQL with reeval query. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. `reeval query` runs read-only SQL over every recorded and scratch run, and computes its answers by fixed rules. [`reeval analyze`](/guide/analyze) explains a run in plain words from the same facts. ```bash npx reeval query ``` ```text history.db: 9 runs Views (the common questions; filter them with SQL): drift(run_id, prev_run_id, eval_key, cell_key, changed, changed_paths, noise, real_changed) effort(run_id, eval_key, cell_key, role, variant, harness, environment, n, tools_median, …) eval_trend(eval_key, run_id, started_at, branch, pr_number, recorded, outcome, pass_at_1, fingerprint) factor_effects(factor, level, baseline, context, tier, run_id, eval_key, strata, runs, …) failing_attempts(run_id, started_at, eval_key, attempt_key, cell_key, harness, model, …) flaky_cells(eval_key, cell_key, fingerprint, runs, flaky_runs, first_seen, last_seen, …) lift(run_id, started_at, branch, eval_key, harness, model, variant, control, …) skill_reads(run_id, eval_key, cell_key, attempt_key, attempt_id, variant, skill, placement, read_line, reads) verdict_flips(run_id, prev_run_id, eval_key, cell_key, judgement_key, question, prev_pass_rate, pass_rate, n) … ``` With no argument, `reeval query` lists the views, the tables and a few examples. A view is a common question with the joins done for you, which you filter with SQL like any table. The history here is the runs from the earlier pages, including the two house-greeting compares. ## Track lift over time ```bash npx reeval query "SELECT eval_key, harness, variant, variant_pass, variant_counted, control_pass, control_counted, points, lo, hi, within_noise FROM lift" ``` ```text history.db: 9 runs eval_key harness variant variant_pass variant_counted control_pass control_counted points lo hi within_noise evals/skill.eval.ts › house greeting › greets in the house… cursor:composer-2.5 with-house-greeting 2 2 0 2 1 0.07 1 0 evals/skill.eval.ts › house greeting › greets in the house… cursor:composer-2.5 with-house-greeting 2 2 0 2 1 0.07 1 0 evals/skill.eval.ts › house greeting › greets in the house… antigravity:gemini-3.8-flash-medium with-house-greeting 2 2 0 2 1 0.07 1 0 ``` Each row is one variant against its control in one run. `points`, `lo` and `hi` are fractions, so `1` is +100 points and `0.07` is +7. `reeval query lift` prints the whole view: reeval query lift in a terminal, one row per run of the house-greeting compare with its lift and interval ## Measure effort ```bash npx reeval query "SELECT eval_key, cell_key, n, tools_median, control_tools_median, tools_ratio, tools_p, verdict FROM effort WHERE role = 'variant'" ``` ```text history.db: 9 runs eval_key cell_key n tools_median control_tools_median tools_ratio tools_p verdict evals/skill.eval.ts › house greeting › greets in the house… cursor:composer-2.5/with-house-greeting/docker 2 3 8 0.375 0.3333 insufficient evals/skill.eval.ts › house greeting › greets in the house… cursor:composer-2.5/with-house-greeting/docker 2 3 11.5 0.2609 0.3333 insufficient evals/skill.eval.ts › house greeting › greets in the house… antigravity:gemini-3.8-flash-medium/with-house-greeting/doc… 2 3.5 24.5 0.1429 0.3333 insufficient ``` `effort` is the result block's effort table as data: median tool calls, agent time and tokens for each cell, and for a variant, its control's. `tools_p` and `tokens_p` are exact rank tests. With two attempts a side they can never drop below 0.05, so those verdicts are `insufficient`, however large the gap. `tokens_verdict` is null when either side has no usage sample: omitted, not insufficient. It's per cell, so a Goose cell, which reports no tokens, doesn't blank it on Cursor cells in the same study. `separated`, `time_separated` and `tokens_separated` are 1 when the variant's and control's ranges don't overlap on tool calls, agent time or tokens. At three attempts a side or more, that's a direction, though the test is still `insufficient`. ## Pool runs to separate the factors ```bash npx reeval query "SELECT factor, level, tier, runs, points, lo, pass_verdict, tools_p, effort_verdict FROM factor_effects WHERE factor = 'environment'" ``` ```text history.db: 9 runs factor level tier runs points lo pass_verdict tools_p effort_verdict environment with-house-greeting controlled 1 1 0.07 insufficient 0.3333 insufficient environment with-house-greeting controlled 1 1 0.07 insufficient 0.3333 insufficient environment with-house-greeting controlled 1 1 0.07 insufficient 0.3333 insufficient environment with-house-greeting pooled 2 1 0.307 effect 0.0286 fewer ``` `factor_effects` asks what one factor (the environment, the harness or the model) changes, and says how good the evidence is. `controlled` compares two cells in one run, and `pooled` combines the same comparison over comparable runs. One run's two attempts a side are too few for either verdict, so each `controlled` row is `insufficient`. Pooling Cursor's two compares gives four attempts a side, enough to call the pass rate an `effect` and the skill's saving `fewer` tool calls. ## Check that the agent read the skill ```bash npx reeval query "SELECT cell_key, skill, read_line, reads FROM skill_reads" ``` ```text history.db: 9 runs cell_key skill read_line reads cursor:composer-2.5/with-house-greeting/docker house-greeting 4 1 … antigravity:gemini-3.8-flash-medium/with-house-greeting/doc… house-greeting 2 1 ``` `skill_reads` has one row per attempt and skill its environment placed. `read_line` is the line in `transcript.jsonl` of the first tool that opened it: a path under `skills//`, or a dedicated `skill` or `load_skill` call. It's null, with `reads` at 0, when the agent never opened it. A variant that passes without reading its skill didn't pass because of it. `expect(turn).toHaveCalledSkill(name)` checks the same signals during the eval. ## See what each arm leaned on ```bash npx reeval query "SELECT cell_key, path, reads, first_line FROM skill_files" ``` ```text history.db: 3 runs cell_key path reads first_line cursor:composer-2.5/skilled/docker ~/.agents/skills/contract-probe/SKILL.md 1 3 claude-code:claude-haiku-4-5/skilled/docker ~/.claude/skills/contract-probe/SKILL.md 1 3 codex:gpt-5.4-mini/skilled/docker ~/.agents/skills/contract-probe/SKILL.md 2 3 ``` `skill_files` lists every file an attempt's environment declared (each skill's `SKILL.md` and references, rules files) and how many tool calls opened it, directly, through a glob or a shell command. A `NULL` `first_line` means the agent never opened it. `context_use` sorts each attempt's tool calls into classes, a heuristic over tool names and paths that `reeval analyze` cites as counts: | Class | Tool calls on | |---|---| | `context` | A placed skill, rules file or MCP tool | | `dependency` | `node_modules`, `site-packages`, `--help` | | `web` | A web search or fetch tool, or `curl` or `wget` of a URL | | `verify` | A check's command, tests, builds | | `workspace` | Everything else in the workspace | An arm that spends its calls in `dependency` while another stays in `context` is digging for what the other was told. ## Count tokens ```bash npx reeval query "SELECT c.role, a.served_model, COUNT(*) AS n, round(AVG(a.input_tokens)) AS input, round(AVG(a.output_tokens)) AS output FROM attempts a JOIN cells c ON c.run_id = a.run_id AND c.eval_key = a.eval_key AND c.key = a.cell_key WHERE c.eval_key LIKE 'evals/skill.eval.ts%' AND c.harness_kind = 'cursor' GROUP BY c.role, a.served_model" ``` ```text history.db: 9 runs role served_model n input output control composer-2.5[fast=true] 4 92569 1458 variant composer-2.5[fast=true] 4 33045 494 ``` The `attempts` table has each attempt's tokens and the model that served it. With the skill, Cursor used about a third of the tokens. `served_model` shows the parameters too: this run's `composer-2.5` ran as `fast=true`. A cell's key repeats across evals in a run, so join `cells` on `eval_key` as well. `reeval query effort` also ranks those totals when both sides of a compare reported usage. ## Script it and rebuild it ```bash npx reeval query "SELECT harness, points, lo FROM lift WHERE within_noise = 0" --json ``` ```text [ { "harness": "cursor:composer-2.5", "points": 1, "lo": 0.07 }, … ] ``` `--json` prints the rows as JSON, and the `history.db:` line goes to stderr, so the output pipes cleanly into `jq` or a script. `--schema` lists every table and column. The database, `.reeval/history.db`, is a cache built from the run records and brought up to date on every query. It's read-only: ```text reeval: reeval query is read-only: .reeval/history.db is a projection of the run records (attempt to write a readonly database) ``` `--rebuild` throws it away and reads every record again. ## Next A checked, plain-words reading of a run. Every view, table and flag.