--- title: reeval mcp description: >- Serve the run history to other agents over MCP, as read-only tools and resources. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. `reeval mcp` is an MCP server built into the CLI. Cursor, Claude Code or any MCP client launches it, asks why a run failed, and asks the history what `reeval query`, `reeval show`, `reeval compare` and `reeval history` answer. ```bash npx reeval mcp npx reeval mcp --http --port 4986 ``` ## Help [//]: # (generated:cli-help:mcp) ```text reeval mcp Serve the run history to other agents over MCP (stdio): read-only tools for why a run failed, reeval query's views and SQL, history, show, compare, transcripts and saved analyses, and runs as resources --http Stateless Streamable HTTP on 127.0.0.1 instead, at /mcp; --port (default 4986), --host --token With --http, require it as a bearer token (also REEVAL_MCP_TOKEN) ``` [//]: # (/generated:cli-help:mcp) ```text reeval mcp · acme · evals/.history/ http://127.0.0.1:4986/mcp Ctrl-C to stop. ``` - **Protocol:** MCP 2026-07-28, stateless (`server/discover`, no sessions). Clients that still open with the 2025 `initialize` handshake are served too. - **stdio** (the default): the client starts `reeval mcp` and it ends when the client closes its stdin. Its stdout is the protocol, so anything the config prints goes to stderr. - **`--http`:** Streamable HTTP, POST `/mcp`, a fresh server per request, on `127.0.0.1` until Ctrl-C. No token by default. A taken port fails; there's no next port. - **Project:** the one `--config` names, or the nearest config to the current directory. ## Tools | Tool | Arguments | Returns | |---|---|---| | `failures` | `run`, `filter`, `limit`, `tail` | Why each failed attempt failed, and whether its cell failed before ([below](#failures)) | | `history` | `filter`, `limit`, `all` | `reeval history --json` without each run's `evals` (`show` has them); with no recorded runs, the scratch runs and a `note` | | `show` | `run`, `format` (`json` or `md`) | `reeval show --json`, or `summary.md` as `markdown` | | `compare` | `base`, `head`, `all` | `reeval compare --json`'s changed rows, and `same`, the number of unchanged evals; `all: true` lists them too | | `analysis` | `run` | Saved and cached analyses of the run, newest first; never calls a model | | `transcript` | `run`, `attempt`, `offset`, `limit` | An attempt's transcript lines (200 by default), `total` and its resource `uri` | | `query_` | `where`, `limit` | `{ columns, rows, truncated }` from the view | | `query` | `sql`, `limit` | The same, for one `SELECT` or `WITH` statement | | `query_schema` | none | `reeval query --schema --json`, as `objects` | There's one `query_` tool per [view](/reference/cli/query#views): `query_lift`, `query_family_lift`, `query_effort`, `query_contrasts`, `query_question_pairs`, `query_factor_effects`, `query_skill_reads`, `query_context_use`, `query_skill_files`, `query_flaky_cells`, `query_drift`, `query_failing_attempts`, `query_eval_trend`, `query_verdict_flips`, `query_check_discrimination`, `query_check_rates` and `query_check_streaks`. Its `where` takes the view's columns, each equal to a value (`null` matches `NULL`). Rows default to 100 and stop at 1000; `truncated` says there were more. A `run` is a [run reference](/reference/cli#run-references), `latest` by default. An `attempt` is its key from `show` (`evals/a.eval.ts › writes ok # fake/-/fake#1`), or its id (`fake/-/fake#1`) when only one eval has it. ```json { "columns": ["eval_key", "run_id", "started_at", "branch", "outcome", …], "rows": [ { "eval_key": "evals/a.eval.ts › writes ok", "run_id": "20260928T064231Z_e404369_cursor-agent_xxa1", "outcome": "pass", "pass_at_1": 1, … } ], "truncated": false } ``` Each result is that JSON as text and as `structuredContent`. Every tool is marked read-only and idempotent. ## failures The server's instructions tell an agent to call `failures` first when a run failed. It returns up to `limit` failed and infra-error attempts (10 by default), gated ones first, each with what a debugger reads first: ```json { "runId": "20261018T165139Z_nogit_cursor-agent_zx6x", "failing": 1, "attempts": [ { "eval": "evals/greet.eval.ts › greeter › creates the greeter", "outcome": "fail", "gated": true, "error": { "kind": "assertion", "message": null }, "failedAssertions": [ { "label": "greet.js exists", "matcher": "toHaveFile", "message": "greet.js exists: expected file src/greet.js to exist", … } ], "failedJudgements": [], "failedChecks": [], "changedFiles": ["added lib/greet.js"], "finalMessage": "Created lib/greet.js.", "transcript": { "lines": 5, "tail": [{ "line": 3, "edit": "lib/greet.js", … }, …] }, … } ], "cells": [ { "cell": "fake/-/fake", "new": true, "failingSince": "20261018T165139Z_nogit_cursor-agent_zx6x", "lastPass": null, "earlier": [], … } ], "next": "For the whole story of one attempt, transcript { … }" } ``` | Field | What it holds | |---|---| | `error` | The attempt's error kind and message; `message` is `null` when it repeats a failed assertion's | | `failedAssertions` | Each `expect` that failed or couldn't answer, with its label, matcher and message | | `failedJudgements` | Each failed judge question, its score and threshold | | `failedChecks` | Each `c.exec` that exited nonzero, and the end of its log | | `setupLogTail` | The end of `setup.log`, for an infra error | | `transcript` | Its line count and the last `tail` lines (12 by default): prompts, messages, edits, and tool calls with their target and output, cut short | | `cells[].new` | `true` when the cell didn't fail in its previous run (or has none) | | `cells[].failingSince` | The run the cell's current failing streak started in | | `cells[].lastPass` | The last run the cell passed cleanly in, and what changed since: fingerprint parts and eval source files | On a run with nothing failed it returns `failing: 0` and a `note` naming the latest run that has a failure. ## Resources | URI | Contents | |---|---| | `reeval://runs/` | The run record; the list has every recorded and scratch run, newest first | | `reeval://runs//transcripts/` | The attempt's `transcript.jsonl`, with the attempt key percent-encoded | ## Security | Guarantee | How | |---|---| | Read-only | SQL runs on the same read-only connection as `reeval query`, one `SELECT` or `WITH` statement at a time, so `ATTACH`, `PRAGMA` and `VACUUM INTO` are refused. The only write is `.reeval/history.db` catching up with new records | | No evals run | A run spends keys, starts containers and takes minutes: run it with `reeval run` in a terminal | | Masked | Records are masked when reeval writes them, and everything `reeval mcp` returns is masked again against its own environment. It loads no `.env` files | | Scoped | Runs are references, never paths. A transcript is read only from inside its run's folder, after resolving symlinks | | Local over HTTP | It binds `127.0.0.1` and checks `Host` and `Origin`, so a web page can't reach it through DNS rebinding. Other hosts and origins get 403; GET gets 405 | ## A token ```bash REEVAL_MCP_TOKEN=$(openssl rand -hex 32) npx reeval mcp --http npx reeval mcp --http --token "$REEVAL_MCP_TOKEN" ``` With `--token ` or `REEVAL_MCP_TOKEN` set (the flag wins), every request needs `Authorization: Bearer `; without it, or with another value, it gets 401. reeval never prints the token, and masks it like any secret. Set one when something other than you can reach the port: - **Beyond localhost:** a `--host` other than `127.0.0.1`, `localhost` or `::1` (reeval warns), or a forwarded port. - **Shared machines:** other users on the same host can reach `127.0.0.1` too. ## Related - [Query history from your agent](/recipes/agent-mcp) - [reeval query](/reference/cli/query) - [reeval history, show, compare](/reference/cli/history)