--- title: Debugging description: >- Find out why an attempt failed, from the result block to the full phase trace in .reeval/debug. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. When an eval fails, the reason is usually in its result block. When it isn't, reeval has kept a trace of every phase of the attempt on your machine. This page goes from the first to the second. ## Read the failure block ```text [greets#1 control] ▶ evals/skill.eval.ts › house greeting › greets in the house style [greets#1 control] +0:00 sandbox ready 33ms [greets#1 control] +0:00 turn 1 done 6ms · 0 tool calls, 2 file edits [greets#1 control] +0:00 $ node -e "import('./greet.js').then((m) => process.exit(m.greet('Ada') ===… exit 1 33ms [greets#1 control] Error: house greeting: expected command to succeed, exit code 1 [greets#1 control] --- stderr --- [greets#1 control] [greets#1 control] --- stdout --- … fail house greeting › greets in the house style arm pass plain 0 of 2 with-house-greeting 2 of 2 can't tell yet check plain with-house-greeting house greeting 0 of 2 2 of 2 expected command to succeed, exit code 1 +2 more passed ``` The attempt's log ends with its error and the command's output; the eval's result block says which check failed, in which arm, and how often. This is the fake agent with `compare.gate: 'all'`, so the control gates too. An ungated control's log keeps only the error's first line, and its misses never fail the run. `summary.md`, under `## Attempts`, links each attempt's folder: its check logs, transcript and diff. ```text .reeval/runs////2/ checks/1.log a c.exec: command, exit code, stdout, stderr diff.patch what the agent changed in the workspace transcript.jsonl every message, tool call and file edit ``` ## Use agent mode under a coding agent ```bash REEVAL_UI=agent npx reeval run results ``` ```text plan run 20261020T211926Z_da3d797_cursor-agent_dcy5 · 5 evals · 5 cells · 6 attempts · ~1s serial (~313ms at concurrency 4) … reeval run: 20261020T211926Z_da3d797_cursor-agent_dcy5 · 1 eval file pass writes greet.js fail greets in French french greeting failed expected command to succeed, exit code 1 +1 more passed … 770ms · results .reeval/runs/20261020T211926Z_da3d797_cursor-agent_dcy5/summary.md 3 fail · 1 pass · 1 infra error · 2 skipped Set analysis.model in reeval.config.ts for answers to what each eval asks. ``` Agent mode prints only the plan, the result blocks and the tally, with the `summary.md` path. reeval picks it on its own when a coding agent runs it, so an agent fixing your evals reads a few lines, not the whole log. `--ui agent` or `REEVAL_UI=agent` turns it on by hand. ## Trace each phase with -v ```bash npx reeval run brokering -v --ui plain --log-order grouped ``` ```text … [agent#1 cursor] 17:40:47 plan harness cursor:composer-2.5 (reeval.config.ts or its default) [agent#1 cursor] 17:40:47 plan sandbox docker (reeval.config.ts or its default) [agent#1 cursor] 17:40:47 plan environment - (none set) [agent#1 cursor] 17:40:47 plan egress proxy (reeval.config.ts or its default) … [agent#1 cursor] 17:40:47 broker CURSOR_API_KEY stays on the host · the gateway swaps it in on POST api2.cursor.sh/auth/exchange_user_api_key [agent#1 cursor] 17:40:47 image cache hit reeval-sandbox:cursor-0.1.0 (reeval's image for cursor; reeval image build rebuilds it) … [agent#1 cursor] 17:40:49 harness turn 1 started: "Run `env | sort > env.txt` and reply with how many lines env.txt has." … [agent#1 cursor] 17:40:49 gateway POST https://api2.cursor.sh/auth/exchange_user_api_key → 200 · allowed · ↑2 B ↓956 B · 76ms · substituted CURSOR_API_KEY (authorization) … [agent#1 cursor] 17:40:52 harness tool bash called (tool_7f8359d2-9ba7-4108-85f9-2deffec470e) … [agent#1 cursor] 17:40:53 harness tool bash returned in 1027ms (tool_7f8359d2-9ba7-4108-85f9-2deffec470e) … [agent#1 cursor] 17:40:53 check $ env; cat /proc/[0-9]*/environ | tr "\0" "\n" → exit 0 · 43ms … [agent#1 cursor] 17:40:55 teardown outcome pass … ``` `-v` prints one line per event, tagged with its phase. The `plan` lines say where each setting came from, which answers most "why did it use that?" questions. `gateway` lines show every request the sandbox made and whether a key was swapped in. ## Find a slow or stuck MCP server ```text [mcp#1 cursor] 09:29:20 env stdio MCP servers start through reeval's MCP timer [mcp#1 cursor] 09:29:20 harness turn 1 started: "Call the house_stamp tool of the stamp MCP server once, …" … [mcp#1 cursor] 09:29:29 harness turn 1 completed in 9s · 1 tool call · 0 file edits · model composer-2.5[fast=false] [mcp#1 cursor] 09:29:29 harness turn 1 mcp stamp answered initialize in 23ms ``` reeval starts every stdio MCP server through a small timer in the sandbox, which notes when the server answers the agent's `initialize`. `-v` prints each start after its turn, and the record keeps it as `turns[].mcpStartup` (see [the record](/reference/record#mcpstartup)). A server that doesn't answer within 60s fails the attempt instead of holding the turn: ```text infra error lands turn error MCP server stamp didn't answer initialize within 60s infra error MCP server stamp didn't answer initialize within 60s ``` To compare start-up times across runs, query the `mcp_startup` table: ```bash npx reeval query "SELECT a.id, m.turn, m.server, m.ms FROM mcp_startup m JOIN attempts a ON a.run_id = m.run_id AND a.key = m.attempt_key" ``` ## See raw payloads with -vv ```bash npx reeval run "passes its own tests" -vv ``` ```text … [passes#1 fake] 17:54:54.930 harness turn 1 started: "Create greet.js and greet.test.js. No dependencies." [passes#1 fake] 17:54:54.930 harness turn 1 prompt [passes#1 fake] {"turn":1,"prompt":"Create greet.js and greet.test.js. No dependencies."} … [passes#1 fake] 17:54:55.026 check $ node --test → exit 0 · 75ms [passes#1 fake] 17:54:55.027 check check output: node --test [passes#1 fake] {"command":"node --test","exitCode":0,"stdout":"TAP version 13\n# Subtest: greets by name\nok 1 - greets by name\n ---\n duration_ms: 0.4… ``` `-vv`, or `--trace`, adds the payload under each line. That includes the resolved config, each transcript event, check output, docker commands, headers and DNS decisions. The terminal shows the first 2,000 characters of each payload. ## Open the debug folder ```text .reeval/debug///// debug.log the -v trace, always written requests.ndjson one line per egress gateway request (Docker only) trace.ndjson raw payloads, for failing attempts or with -vv ``` | File | Written | Holds | | --- | --- | --- | | `debug.log` | Every attempt | The `-v` lines, with or without `-v` | | `requests.ndjson` | Every attempt that makes a request | Method, URL, status, bytes, key swaps | | `trace.ndjson` | Failing attempts, or every attempt with `-vv` | The `-vv` payloads | So you don't need to rerun with `-v` after a failure: the trace is already there. Everything is redacted before it's written. The trace buffer is capped at 8 MB per attempt, and the oldest entries go first. `.reeval/debug` stays on your machine. `reeval record` never copies it into the history folder, and `reeval studio build` never includes it. ## Use the studio's Debug tab ```bash npx reeval studio ``` The studio's Debug tab for one attempt, showing the phase trace and gateway requests `reeval studio` adds a Debug tab to each attempt that has local debug data. It shows a timeline of the phases, the network requests and the raw trace, next to the transcript and diff. ## Debug reeval itself ```bash REEVAL_DEBUG=1 npx reeval run hello ``` `REEVAL_DEBUG=1` is for when reeval misbehaves rather than the agent. It streams the output of the test engine reeval runs your evals in, plus the Docker sandbox and image build logs, to stderr. It also prints the stack for unexpected errors. If reeval itself is at fault, `npx reeval report --issue` collects a redacted report (versions, `reeval doctor`, the last error and the run's debug logs), writes it locally for you to review, and opens a pre-filled issue: see [reeval report](/reference/cli/report). ## Clean up ```bash npx reeval clean --keep 1 ``` ```text Removed debug data for 4 runs (106.9 KB) from .reeval/debug ``` reeval keeps debug data for the newest 20 runs and removes older ones as it goes. `reeval clean` removes it all now, or all but the newest `--keep` runs. Change the defaults in the config: ```ts title="reeval.config.ts" export default defineConfig({ debug: { keep: 50, traceBufferMb: 16 }, // … }) ``` ## Next Every file and field in `.reeval/debug`. What each error means and how to fix it.