--- title: Explaining a run description: >- Get a checked, plain-words reading of a run with reeval analyze, and insights on each eval at the end of a run. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. `reeval analyze` makes one LLM call over the facts `reeval query` computes and explains the run in plain words, with every claim checked against those facts. ```bash npx reeval analyze --config reeval.agents.config.ts --save ``` ```text analysing 20260926T174444Z_b010670_ada-lovelace_7gmj vs 20260926T174239Z_b010670_ada-lovelace_yzmg with google/gemini-3.8-flash… … Results - For eval 'skill', the house-greeting skill produced a controlled pass-rate lift of +100 pts (95% CI +7…+100, verdict: effect) in both cursor:with-house-greeting and antigravity:with-house-greeting relative to their respective control cells. [F5… … - Within run r2 alone, effort reductions on eval 'skill' (cursor: 3 vs 11.5 tool calls; antigravity: 3.5 vs 24.5 tool calls; p = 0.3333 for both) had a verdict of insufficient because 2 vs 2 attempts cannot reach statistical significance. [F10] … Suggested actions 1. Increase attempt count on skill eval to reach statistical power for effort (eval · impact medium · effort low) [F10] [F11] target: evals/skill.eval.ts. Run more attempts per cell (at least 6 per cell) so exact Mann-Whitney effort tests can achieve p < 0.05 significance rather than returning insufficient. expect: Effort differences (fewer tool calls and agent seconds) reach statistical significance instead of insufficient. verify: reeval run --runs 6 --tag skill … gemini-3.8-flash · analysis-v2 · digest v1 · 10381 in / 3196 out tokens · 13s · 29 citations checked, 0 invalid, 0 dropped · 1 commands checked, 0 rejected ``` It explains the latest run against its nearest recorded ancestor, here the cross-agent run from [Harnesses](/guide/harnesses): what happened, the results, why, anomalies, and at most five actions. `--vs none` writes up this run alone, with no drift, eval-changed or cross-run factor rows. ## What reeval checks | Check | What happens when it fails | |---|---| | Every claim cites a fact (`F…`) or evidence (`E…`) from the digest | The claim is dropped | | A number in a claim is in a cited fact, anomalies included | The number is flagged | | A quoted judge score is the cited attempt's recorded score | The line is dropped | | A verify command parses against reeval's CLI, and its `--tag` is one the digest lists | The action is dropped | reeval computes the facts and the model only explains them. After the checks it caps confidence: not `high` when any citation or command failed, and `low` when a claim was dropped or a command was rejected. The TL;DR is one sentence (what happened, how sure, why, next) and shows that capped value. An `insufficient` lift or effort verdict isn't measured, so it's never a results line. When the eval's sources changed, the digest says `not comparable` and drops the mixed pass rate. A [question](/guide/questions)'s answers are in the digest as annotation: they never gate pass, fail or the exit code. A `--tag` that no eval has, or any other rejected verify, drops the action. It isn't printed with `verify: none`. ## See it in the studio reeval studio Analysis tab with the TL;DR, what happened and results, each claim followed by its citations The studio shows saved analyses on the run's Analysis tab, with citations that link to attempts. The run page leads with a deterministic pass, fail and infra sentence, and pins the TL;DR under the header when an analysis exists. ## Check it, cache it, save it ```bash npx reeval analyze --config reeval.agents.config.ts --dry-run ``` ```text digest 20260926T174444Z_b010670_ada-lovelace_7gmj vs 20260926T174239Z_b010670_ada-lovelace_yzmg: 24 facts, 5 evidence items, 18 attempts size 26510 characters ≈ 6628 tokens (cap 20000 in, 8000 out) model google/gemini-3.8-flash · analysis-v2 · cache miss (.reeval/analysis/cb4f1ade460a94fdd4e5a9ddef213bd995e81adbbf4f83d0c4a7fa93e0a69398.json) ``` | Flag | What it does | |---|---| | `--dry-run` | Shows the digest's size and the cache key, without a call | | `--no-cache` | Asks again: otherwise `.reeval/analysis` answers the same inputs without a call | | `--save` | Also writes the analysis into the run's history folder as JSON and Markdown, to commit | | `--md`, `--json` | Prints it for a PR comment or a script | | `--context` | Prints the digest and reeval's rules as Markdown for your own coding agent, with no key and no call | ## Choose the model ```ts title="reeval.config.ts" export default defineConfig({ analysis: { model: 'google/gemini-3.8-flash', // default: the judge's, if it can instructions: 'Fewer tool calls count as a win.', }, // … }) ``` `instructions` opens the prompt, before the digest, as the question the analysis answers. A run where every cell has `gate: 'none'` is read as a study: the TL;DR says how the arms differ, check by check, rather than that attempts failed. ## Insights during a run ```ts title="reeval.config.ts" export default defineConfig({ analysis: { model: 'google/gemini-3.8-flash' }, // … }) ``` ```text insights: 11 evals with google/gemini-3.8-flash (Ctrl-C skips the rest; --no-insights skips them all) pass greet module › adds a default name in a follow-up turn Yes, the agent added the default parameter without breaking existing functionality in 1 of 1 tries. … All 16 results passed across greeting, code generation, and framework recommendations. 16 pass ``` With `analysis.model` set, `reeval run` asks the model what each eval checks and how the run answers it, once the attempts finish and the exit code is fixed. `reeval init` sets it to `google/gemini-3.8-flash`, which needs `GOOGLE_GENERATIVE_AI_API_KEY`. An `anthropic/*` or `openai/*` model takes `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` the same way, and any model goes through Vercel AI Gateway once the config lists [`vercel()` in `gateway`](/reference/gateway#the-judge-analysis-and-captures). Each sentence cites the run's facts and passes the [insight checks](/concepts/insights#every-sentence-cites-a-fact), or it's dropped. | Setting | Default | What it does | |---|---|---| | `analysis.model` | none | The model; unset, a run prints no insights | | `analysis.maxTokens` | `250000` | Caps one run's insight tokens; failing evals, regressions and compares go first, then passes with a failed try | | `analysis.insights`, `--no-insights` | on | Turns them off, for every run or for one | Insights run to the end, with a progress line on a terminal (`insights 3/11…`). They never change an outcome or the exit code. Past the budget, when an eval's calls fail twice, or after Ctrl-C, those blocks print their facts and one line says why, here from a project of greeting evals on the fake harness with `analysis.maxTokens: 12000`: ```text insights skipped for 4 evals: analysis.maxTokens (12k) reached — raise it in reeval.config.ts, or reeval analyze 20261021T182958Z_a343e84_cursor-agent_bfvz --insights to finish ``` ```bash npx reeval analyze 20261021T182958Z_a343e84_cursor-agent_bfvz --insights \ --config reeval.fake.config.ts ``` ```text finishing insights for 20261021T182958Z_a343e84_cursor-agent_bfvz with fake: 4 evals left… wrote .reeval/runs/20261021T182958Z_a343e84_cursor-agent_bfvz/insights.json: 11 evals, 0 sentences removed by the checks. Commit it with git. … ``` That command asks about only the evals left, with a fresh budget, and keeps every sentence already written. Without the model's key, the plan warns up front, and the run prints `insights skipped: set GOOGLE_GENERATIVE_AI_API_KEY, or analysis: { insights: false }` and its facts; `reeval analyze --insights` and `reeval doctor` say every way to set it. [Insights and remarks](/concepts/insights) covers the headline and the remarks that end a run. ## Next What a sentence is, how it's checked, and the remarks. Find out why an attempt failed. Every flag, the digest and the checks.