---
title: Errors
description: Common reeval error messages, what each one means, and how to fix it.
---
> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

reeval errors name what's wrong, where it was set and the fix. This page groups the common ones by when they happen.

```text
reeval: Unknown option "harnesses" in reeval.config.ts. Did you mean "harness"?
reeval: judge.threshold in evals/a.eval.ts › e must be between 0 and 1 (got 2)
```

## Reading an error

| Part | Example | What it tells you |
|---|---|---|
| The field | `judge.threshold`, `profiles.pr.limit.runs`, `files["a.txt"]` | The option path, as you'd write it in code |
| Where | `in reeval.yaml`, `in evals/a.eval.ts › e`, `in defineEnvironment("base")` | The config file, eval or helper call that set it |
| The problem | `must be between 0 and 1`, `is missing`, `has an unknown key "rf"` | What the value should be |
| What you passed | `(got 2)` | The value reeval saw, as JSON |
| The fix | `Did you mean "ref"?`, `Set it in the environment or .env.local.` | The next step, when there's one to name |

## When they happen

| When | Exit code | What it means |
|---|---|---|
| Loading the config and eval files, planning | 2 | Nothing ran. Fix the config, the eval file or the command. |
| Preflight: credentials, Docker, images, budget | 2 | Nothing ran. Set the key, start Docker, build the image, or narrow the run. |
| During an attempt: a check fails | 1 | The agent's work didn't pass. That's a result, not a bug. |
| During an attempt: reeval or the sandbox fails | 3 | An infra error: the attempt couldn't be judged. It doesn't count as a pass or a fail. |
| Ctrl-C | 130 | Running attempts are cancelled. |
| `SIGTERM` (`docker stop`, a cancelled CI job) | 143 | Running attempts are cancelled. |

Errors in an eval file are prefixed with the file (`reeval: evals/skill.eval.ts: …`). See [reeval run](/reference/cli/run) for exit codes in full.

## reeval doctor

`reeval doctor` runs the same checks before a run, and puts each problem's fix on the line under it:

```text
✗ config     Unknown option "limits" in profiles.pr in reeval.config.ts. Did you mean "limit"?
             fix: edit reeval.config.ts, then reeval run --dry-run
✗ key        CURSOR_API_KEY for cursor:composer-2.5: not set
             fix: add CURSOR_API_KEY=... to .env.local
…
2 problems, 0 warnings. Fix them and run reeval doctor again.
```

See [reeval doctor](/reference/cli/doctor) for every check.

## Config and options

| Message | Fix |
|---|---|
| `Unknown option "repeat" in evaluate("e"). Did you mean "runs"?` | Use the suggested key. See [Options](/reference/options). |
| `compare can't be set in reeval.config.ts: it would multiply every eval. Put it on the group or eval that asks the question.` | Move `compare` to a group or eval; narrow it per profile with `limit.compare`. |
| `env.API_KEY in reeval.config.ts is a literal string. Pass secrets with secret.fromEnv() so they're redacted and never recorded.` | Use the callback form and `secret()` or `secret.fromEnv()`. |
| `files["../x"] in defineEnvironment("e"): paths are relative to the workspace, or start with ~/ for the home directory, and never use ..` | Write the file inside the workspace or home; an absolute path or `..` would land outside the sandbox's folders (on the fake sandbox, on your machine). Backslashes get `use forward slashes (docs/api.md)`. |
| `judge.model in reeval.config.ts must be a "provider/model" string or fakeJudge(). Pass a model object per group, eval or call instead.` | Use a model string in the config. |
| `judge.model in reeval.yaml needs a provider prefix, e.g. "typesafe-ai/jev-1.14.0" or "google/jev" (got "jev")` | Name the provider before the model. |
| `Unknown judge option "diff" in reeval.config.ts. Did you mean "state.diff"?` | Use the suggested key. See [Judges](/reference/judges#what-the-judge-is-given). |
| `Unknown judge.state key "transcipt" in reeval.config.ts. Did you mean "transcript"?` | Use the suggested key. A bad value names what the key takes, such as `judge.state.turns in reeval.config.ts must be 'last' or 'all' (got "first")`. |
| `judge.providerOptions maps each provider to an object of JSON values, …` | Key `providerOptions` by provider, each an object: `{ google: { thinkingConfig: { thinkingBudget: 0 } } }`. |
| `The defineConfig callback in reeval.config.ts returned a Promise. …` | Make the callback synchronous. |
| `Config file not found: nope.ts` | Check the `--config` path. |
| `No reeval.config.ts (or reeval.config.mts, reeval.config.cts, reeval.config.js, reeval.config.mjs, reeval.config.cjs, reeval.yaml, reeval.yml) here or above. Run reeval init.` | `reeval run` and `reeval image build` need a config file. Run `reeval init`, or pass `--config`. A parent-directory config still counts. |
| `reeval.config.ts and reeval.yaml are both in .: reeval reads one config per directory. Keep one, or pass --config.` | A directory has a script config and a `reeval.yaml` (or both `reeval.yaml` and `reeval.yml`). Delete one, or name the one to read with `--config`. |
| `sandbox.dockerfile in reeval.yaml: /project/docker/Dockerfile doesn't exist. Paths in reeval.yaml are relative to its directory.` | A relative path in a config, `reeval.yaml` or `reeval.config.ts`, resolves against the config's directory, not the one you run `reeval` from. Fix the path, or add the Dockerfile there. The same Dockerfile written as data in `defineEnvironment` says `sandbox.dockerfile in defineEnvironment("e"): … Paths are relative to the file that defines the environment.`, and in an eval's options, TypeScript or a declaration tree from another language, `sandbox.dockerfile in evaluate("e"): … Paths are relative to the file that declares it.` |
| `reeval.config.ts has no default export (it exports config). Write export default defineConfig({…}).` | Export the config as the default: `export default defineConfig({ … })`. A default export that isn't a config gets `reeval.config.ts's default export must be defineConfig({…}) (got …)`. |
| `Unknown option "limits" in profiles.pr in reeval.config.ts. Did you mean "limit"?` | Use the suggested key. The same goes for `budget`, `debug`, `secrets`, `analysis` and `defaultProfile`; without a close match, the message lists the keys. |
| `Invalid duration "10 minutes". Use e.g. "90s" or "15m".` | Use `ms`, `s`, `m` or `h`, or a number of milliseconds. |
| `Invalid duration 0. A duration is more than zero: …` | A timeout or budget of zero (or less) would stop every attempt at once. Give a positive duration. |
| `evaluate("e") is declared twice in this file. …` | Give each eval its own name. For `evaluate.each`, put `$key`, `%s` or `%#` in the name. |
| `evaluate.each(cases, options) takes { pool?: boolean } (got …). Eval options such as compare go after the name: …` | Move the eval's options after the name: `evaluate.each(cases)(name, { compare }, fn)`. |
| `evaluate.each takes an array of cases (got …)` | Pass an array: `evaluate.each([{ n: 1 }, { n: 2 }])`. |
| `Unknown option "pool" in evaluate("x"). pool is an option of evaluate.each, beside the cases: …` | Pass `pool` to `evaluate.each(cases, { pool: false })`, not to the eval. |
| `evaluate("x") got two functions. Options must be a plain object; …` | Options are a plain object. Put secrets in an environment or the config's `env`. |
| `sandbox takes one sandbox, not a list (…). Different toolchains need different checks, so write a separate eval per sandbox.` | One sandbox per eval. |
| `environment takes one environment, not a list (evaluate("x")). To compare environments, use compare: { baseline, variants: [a, b] }. baseline is the control; variants are not layered on it.` | Use `compare`. |
| `compare in evaluate("e") must be an object, not a list. Use compare: { baseline, variants: [...] }.` | Pass `{ baseline, variants }`, not a bare array. |
| `compare.baseline in evaluate("e") must be an environment from defineEnvironment (got "a"). …` | Set `baseline` to an environment, or leave it out to make the eval's environment the control. Variants are not layered on it. |
| `compare in evaluate("g › e") lists a, which is the control: compare has no baseline, so the eval's environment is. …` | Drop that variant: without a `baseline`, the cascaded `environment` is already the control. |
| `compare in evaluate("e") lists a variant named baseline, the name of the empty control …` | Rename the variant, or set `compare.baseline`. |
| `compare in evaluate("e") runs head and base on the same tree: past what they share, each sets only workspace, and both are git("..") with no ref (the repository HEAD). …` | The two arms can only run the same code. A `ref` read from an unset variable is `undefined`: set the variable, or give each arm its own ref. See [Lift of a pull request](/recipes/pr-lift). |
| `Unknown compare option "strategy" in evaluate("e")` | Drop `strategy`. Share setup with `extendEnvironment` before `compare`. |
| `environment and compare.baseline can't both be set in evaluate("e")` | Drop `environment`, or drop `baseline` to make `environment` the control. A parent `environment` can stay beside a child's `baseline`, and isn't applied. |
| `compare: false can't be set in evaluate("e")` | Omit `compare`, or pass `{ baseline, variants }`. A profile uses `limit.compare: false`. |
| `questions in evals/a.eval.ts › e is a list (got "Which?"), …` | Wrap the question in a list, `questions: ['Which?']`. An empty string, an entry without its `question`, an unknown entry key and the same question twice are refused too; an empty list asks none. |
| `questions "Q" in evals/a.eval.ts › e: across is 'environment' (vary the environment, same harness) or 'harness' (vary the harness, same environment), or a list of both (got "cell"). …` | Use `'environment'`, `'harness'` or a list of both, or leave `across` out to ask each attempt. An empty list and a repeated factor are refused too. |
| `questions "Q" in evals/a.eval.ts › e varies the harness (across: 'harness'), but evals/a.eval.ts › e runs one harness (cursor), so it has no pairs to ask. …` | Run the eval on two harnesses, or drop `across` to ask each attempt. `across: 'environment'` without a `compare` is refused the same way. A profile's limit that leaves one level drops the question with a note instead. |
| `questions "Q" in evals/a.eval.ts › e sets both across and choices. …` | `choices` are for a question asked of each attempt. Drop `choices` to vote between pairs, or `across` to ask each attempt. |
| `questions "Q" choices in evals/a.eval.ts › e is a list of at least two answers, …` | Pass two or more distinct, non-empty strings, e.g. `choices: ['yes', 'no']`. |
| `questions "Q" state.contents in evals/a.eval.ts › e can't be set: questions are asked after the attempts, when their workspaces are gone. …` | Narrow `state.diff` to the files the question is about. |
| `questions "Q" in evals/a.eval.ts › g › e shows the model nothing: every state key is off and it sends no artifacts. …` | Turn a `state` key back on, such as `diff: true`, or send `artifacts`. The cascaded `question` option counts. |
| `Unknown question option "across" in evals/a.eval.ts › e. across goes on each questions entry, …` | `question` takes `model`, `state` and `artifacts`; `across` and `choices` go on the entry. |
| `questions can't be set in reeval.config.ts: each is about one eval's task. …` | Put `questions` on the group or eval they ask about. `question: { model, state, artifacts }` in the config sets every question's defaults. |
| `warning: question "Q" shows state.transcript, which names each harness's tools, so its harness pairs aren't blind. …` | A warning, not an error. Ask it with `across: 'environment'`, or leave the transcript out. |
| `question "Q" artifacts names a capture no paired attempt has: "sign-in"` | On the answer's `error` after the run, with no votes. Check the name passed to `c.capture()`. `state.checks` naming a command no paired attempt ran fails the same way. |
| `questions "Q" in evals/a.eval.ts › e names typesafe-ai/jev-1.14.0, an evaluation model: …` | Give the question a language model, e.g. `'google/gemini-3.8-flash'`. A model without a provider prefix is refused too. |
| `"image" isn't an environment field (defineEnvironment("e")). …` | Image content belongs to `docker({ from, install, dockerfile })`; per-attempt steps to `setup`. The hint names `from` for a base image string and `install` for a list of steps. |
| `Unknown cursor() option "modle". Did you mean "model"?` | Every harness factory refuses a key it doesn't take. Use the suggested one. See the harness's page under [Harnesses](/reference/harness/cursor). |
| `call.options.artifacts[0].file is missing: it must be an object` | Data from another language that misses a field, or has one of the wrong type, names the field's full path. A value that can take several forms is explained against the form it came closest to. |
| `harness in evals/a.eval.ts › e: Unknown cursor() option "modle". …` | A harness, sandbox, secret, skill, MCP server, file, workspace or environment built as plain data (by hand, or by an SDK in another language) is checked as its helper's result is, and named by where it sits: `sandbox in …`, `env.A_KEY in …`, `skills[0] in …`, `mcp.s in …`, `files["a.txt"] in …`, `workspace in …`. Build it with the helper, or fix the named field. |
| `Unknown field "setpu" in layer "x" of environment "x" in evals/a.eval.ts › e.` | An environment as data has each layer's spec checked as `defineEnvironment` checks it. Fix the named field. |
| `evaluate("x") has no function. …` | Pass the eval's body: `evaluate("x", async (c) => { … })`. |
| `evaluate.only isn't allowed in CI` | Remove the `.only` before pushing. |

## Profiles and budgets

| Message | Fix |
|---|---|
| `Unknown profile "nigthly". Profiles in reeval.config.ts: smoke, nightly.` | Check the name in `--profile` or `REEVAL_PROFILE`. |
| `this run plans 220 attempts, over budget.attempts 12 (profile smoke). Narrow it with --tag or --runs, or raise the budget. (--force runs it anyway)` | Narrow the run, raise `budget.attempts`, or pass `--force`. |
| `defaultProfile.ci in reeval.config.ts names "pr", which isn't a profile (profiles: smoke, nightly). Add it under profiles, or name one that's there.` | Fix the name, or add the profile. |
| `profiles.pr.limit.runs in reeval.config.ts must be a whole number of at least 1 (got 0)` | Every profile key names its path and file the same way. |
| `profiles.pr.budget.usd in reeval.config.ts isn't supported yet: reeval records the tokens harnesses report, but prices none and enforces no token budget. Cap the run with attempts or duration.` | Cap with `attempts` or `duration` (`budget.tokens` gets the same message). |

See [Profiles and budgets](/reference/profiles).

## Command line

| Message | Fix |
|---|---|
| `No evals matched evals/hello.eval.ts, … with filter "nope"` | Check the filter: it matches file paths, eval names and cell keys. |
| `filter "x" matches no eval file, eval name or cell in …` | One of several filter words matched nothing. |
| `No eval files in /path/to/project match include: evals/**/*.eval.ts. …` | No file matches `include`: check the file names, or set `include` in `reeval.config.ts`. |
| `3 eval files failed to load:` then `evals/a.eval.ts, …: <error>` | A file threw, imported something missing, had a syntax error (with its position and a code frame), or its `tsconfig.json` isn't valid JSON or extends a file that isn't there (as in a copied example). Nothing runs until every file loads. |
| `cursor: session did not become ready within 90s (tried twice)` | The harness's session start (its bootstrap) hung twice. The attempt is an infra error, its sandbox is destroyed and the other attempts carry on. Raise the harness's `startTimeout` if it's legitimately slow. |
| `The worker running evals/a.eval.ts exited unexpectedly (signal SIGKILL)` | The process running that file died: killed, or out of memory. Its unfinished attempts fail and the run exits 1. |
| `The reeval at <path> runs eval files with engine protocol N, but this reeval speaks M` | The project resolves another version of reeval than the `reeval` you ran. Run the project's own: `npx reeval`. |
| `--runs must be a whole number of at least 1 (got 0)` | Pass a positive whole number. |
| `--json doesn't take a value: pass --json on its own.` | Drop the `=…` after a yes/no flag. A flag that needs a value and has none gets `--runs needs a value (--runs <n>)`. |
| `No run matches "abc". Pass latest, latest~N, a run id or a git SHA; reeval history lists them.` | `reeval show`, `compare` and the other history commands take one of those. |
| `Unknown command "image foo". Did you mean reeval image build?` | Use `reeval image build`. |
| `Unknown command "frob". Run reeval --help.` | See [CLI](/reference/cli). |
| `Refusing to record <run>: 1 file(s) look like they contain a secret:` then `<run>/…/transcript.jsonl line 4: the value of DEPLOY_TOKEN` | A file still holds a secret after redaction, often because the variable was set only after the run. Check the named line. If it's a false positive, `reeval record --skip-secret-scan`. See [the secret scan](/reference/cli/record#the-secret-scan). |
| `warning: left worktree.patch out of <run>: it looks like it holds a secret.` then `worktree.patch: .env.example line 3: Google API key (AIza…tUvW)` | Your uncommitted changes at the start of the run, not the agents' work, hold a key-like value. The run is recorded without the patch, so nothing was committed. Run `git status`, then move the key to an ignored file such as `.env.local`, or ignore the file. Rotate the key only if it's already been committed or pushed. `reeval doctor` warns about keys in `.env` files git sees. |
| `<run>/worktree.patch is missing, but the record says the run's uncommitted changes are in it. …` | `reeval checkout` can't find the patch the record names. Restore it from git, then try again. No worktree is created. |
| `warning: skipped evals/.history/<run>.json: not a readable reeval run record (…). Restore it from git, or delete it.` | A record was cut short or left with merge conflict markers. Every history command carries on without it; restore it with `git checkout -- <file>` or delete it. |
| `No resumable run. A stopped staging directory, or an interrupted or budget-stopped scratch record, can be resumed.` | Nothing on disk is Stopped staging or an `interrupted` / `budget` scratch, and nothing is still running. `reeval run --resume` needs one of those; a complete `latest` is not enough. See [Resume a stopped run](/reference/cli/run#resume-a-stopped-run). |
| `run <id> is still running. Wait for it to finish or stop, then reeval run --resume <id>.` | That id's process is still alive (pid check, including a sleeping laptop), or another `reeval run --resume` already claimed it. Stop that process, or wait, then resume. Omitted `--resume` names the id; it does not say there is no resumable run. |
| `run <id> is complete and cannot be resumed. reeval run starts a new run.` | A finished run keeps its id. Start a new run, or pass a Stopped / interrupted / budget id. |

## Credentials

| Message | Fix |
|---|---|
| `Missing credentials: CURSOR_API_KEY; TYPESAFE_AI_API_KEY (for direct()) for the judge model, "typesafe-ai/jev-1.14.0". Set them in the environment or .env.local.` | Set the harness key and the judge's provider key, in the shell or `.env.local`. With `AI_GATEWAY_API_KEY` set, the message adds `AI_GATEWAY_API_KEY is set, but no gateway in the config uses it: add vercel() to gateway …`: add [`vercel()` to `gateway`](/reference/gateway) to use it. |
| `Credentials with whitespace or a line break in them: CURSOR_API_KEY (character 4). A key copied from a wrapped terminal line picks these up; set it again as one line.` | Set the key again as one line. |
| `Secret NAME for <where> isn't set. Set it in the environment or .env.local.` | Set the host variable, or mark the secret `optional: true`. |
| `warning: SENTRY_DSN is in the sandbox as its real value. …` | Add `hosts` to broker it, or accept the warning. `secrets: { unbrokered: 'error' }` makes it an error. |
| `warning: the egress gateway refused POST https://…: it carries CURSOR_API_KEY's placeholder and isn't one of the endpoints reeval brokers it for` | The harness CLI called an endpoint reeval doesn't broker. Check for a CLI update and report it. |
| `The API rejected CURSOR_API_KEY: set a valid key in the environment or .env.local. …` | The harness's API refused its key (an HTTP 401, or Google's "API key not valid"). The named variable is the one to fix; the provider's own message follows. |
| `No credentials for judge model "…" (provider "…"). Set …` | Set the provider's own key (`ANTHROPIC_API_KEY`, `GOOGLE_GENERATIVE_AI_API_KEY`, `OPENAI_API_KEY` or `TYPESAFE_AI_API_KEY`), or add `vercel()` to `gateway` and set `AI_GATEWAY_API_KEY`, with a decision model such as Jev as the judge there. See [Judges](/reference/judges#providers-and-keys). |
| `the judge model "anthropic/claude-sonnet-5" is a language model, and Vercel AI Gateway judges only with decision models. …` | Judge with a decision model on `vercel()`, such as `typesafe-ai/jev-1.14.0`, or pin the judge to `direct()` with `judge: { gateway: 'direct' }`. See [Gateways](/reference/gateway#the-judge-analysis-and-captures). |
| `the judge model "openai/gpt-5.5" isn't a decision model: reeval's openai/* judges call OpenAI's Decisions API, which answers with 'gpt-6-luna' …` | Judge with `openai/gpt-6-luna`. |
| `questions "Which is cleaner?" (evals/q.eval.ts › asks) needs a language model to answer it. …` | Set `analysis.model`, give the question its own `model`, or set `question: { model }` on its group or eval. A judge that's an evaluation model can't answer it; `--dry-run` still plans. |

## Docker and images

| Message | Fix |
|---|---|
| `Docker is not running. Start Docker and try again.` | Start Docker. On Windows, Docker Desktop on the WSL 2 backend. |
| `Docker isn't installed: there's no docker command on the PATH. …` | Install Docker (Docker Engine or Docker Desktop), then `reeval image build`. |
| `Docker is running Windows containers, but reeval's sandboxes are Linux containers. …` | Switch Docker Desktop to Linux containers. |
| `Docker image … not found. docker({ image }) is used as is, so pull or build it first …` | Pull or build your image, or use `from` or `dockerfile` so reeval builds it. |
| `… The sandbox <id> is set by reeval.config.ts for every eval, so nothing ran.` | The config's sandbox failed to prepare (exit 3); fix the failure above it. |
| `✗ <sandbox> failed to prepare (set by …; only its attempts fail)` | A group's or eval's sandbox failed; its attempts are infra errors with this headline. |
| `Building the toolchain layer FROM … failed: reeval needs a Debian or Ubuntu based image (glibc and apt-get)` | Use a Debian or Ubuntu base for `from` or `dockerfile`. |
| `Building the install layer (N commands) failed: …` | Fix the failing `install` command; its last lines follow, and the build log is in `.reeval/cache/images/`. |
| `Building <Dockerfile> failed: …` | Fix the Dockerfile; the failing step's last lines follow. |
| `out of disk space (ENOSPC): …` | Free space for Docker (`docker system df`), then rerun. |
| `killed (exit 137): usually out of memory; …` | Give Docker more memory, or lower `--concurrency`. |
| `Baking the <harness> CLI onto <base> failed: …` | Usually the network; reeval already retried it twice. |
| `Can't keep CURSOR_API_KEY out of the sandbox: credential brokering could not start (…). …` | The image has no Node for the egress gateway. Use `docker({ from })` or `docker({ dockerfile })`. |
| `Can't enforce egress proxy: the egress gateway, the sandbox's only way out, could not start (…). …` | The same, for any policy but `'allow-all'`: the container has no network without the gateway. Use `docker({ from })` or `docker({ dockerfile })`. |
| `The Cursor harness needs the Docker sandbox: set sandbox: docker() for this eval, or use fakeHarness() with fakeSandbox()` | Real harnesses run only in `docker()`. |

See [docker()](/reference/sandbox/docker).

## Git workspaces

Before any attempt, reeval fetches each `workspace: git(…)` repository on the host. A failure is reported once, like a sandbox image that failed to prepare, and fails only the attempts that use the workspace, as infra errors (exit 3); the other evals run:

```text
✗ workspace https://github.com/vitejs/vite#v99.0.0:packages/create-vite/template-react-ts failed to prepare (set by environment react-ts; only its attempts fail)
  Workspace git("https://github.com/vitejs/vite#v99.0.0:packages/create-vite/template-react-ts"): the repository has no ref "v99.0.0" (a branch, tag or commit SHA).
…
infra error      adds a reset button
  infra error      Workspace git("https://github.com/vitejs/vite#v99.0.0:packages/create-vite/template-react-ts"): the repository has no ref "v99.0.0" (a branch, tag or commit…

pass             writes a README
…
1 pass · 1 infra error
```

Each such attempt's `error.prepare` has `stage: "workspace"` and the workspace as its `ref`.

| Message | Fix |
|---|---|
| `Workspace git("https://github.com/vitejs/vite#v99.0.0:…"): the repository has no ref "v99.0.0" (a branch, tag or commit SHA).` | Check the ref. A commit needs its full 40-character SHA; an abbreviated one gets that hint. A local repository that's a shallow clone gets `… is a shallow clone, so it may not have that ref: fetch it (git fetch origin …) or clone the full history (fetch-depth: 0 in actions/checkout).` |
| `warning: compare in evals/pr.eval.ts › lift: head and base start from the same tree (4ea35bfc6bef): git("..#v2") and git("..#e55bf01…") resolve to it, so the lift can't show a change to the workspace. …` | Two arms' different refs or paths resolved to one tree, so the run measures no change to the code. Check the refs the arms read; the run still goes ahead. |
| `Workspace git("…#v6.0.0:packages/create-vite/template-react-tsx") at v6.0.0 (commit 814120f) has no directory "packages/create-vite/template-react-tsx"; packages/create-vite/ has: …, template-lit-ts/, …` | Pick one of the listed directories. A file gets `is a file, not a directory`. |
| `Workspace git("https://github.com/acme/no-such-repo#v1"): couldn't fetch it (remote: Repository not found.). Check the URL; a private repository also needs this machine's git credentials.` | Check the URL, or give this machine access to the repository. |
| `Workspace git("…"): isn't readable with this machine's git credentials, or doesn't exist (could not read Username for 'https://github.com': terminal prompts disabled). …` | reeval never prompts. Give `git()` a `token`, or run `gh auth setup-git`, configure a credential helper or load an SSH key on the host; the sandbox never gets them. With a `token`, this means the token was refused. |
| `Workspace git("…"): its token variable GITHUB_TOKEN isn't set. Set it in the environment or .env.local.` | Set the variable `token` names. |
| `Workspace git("https://gitlab.invalid/acme/app#v1"): couldn't be fetched (unable to access 'https://gitlab.invalid/acme/app/': Could not resolve host: gitlab.invalid)` | The network, after two retries (its `class` is `transient`). A pinned SHA fetched before reruns from the cache offline. |

## Plugins

The plan reads every local plugin and checks each cell's harness can take it, so a plugin reeval can't place exits 2 before any attempt starts, naming the eval and the harness:

```text
reeval: evals/review.eval.ts: evaluate("reviews the diff") on goose:gemini-3.8-flash: Goose can't load the review plugin's agents (agents/) and commands (commands/) and hooks (hooks/hooks.json) and rules (rules/): use only: ['skills', 'mcp'], or leave the plugin out of this cell
```

| Message after the prefix | Fix |
|---|---|
| `Claude Code can't load the review plugin's rules (rules/): use only: ['skills', 'mcp'], or leave the plugin out of this cell` | The harness doesn't load that part. Leave it out with `only`. |
| `Codex can't load the review plugin's apps: they need a ChatGPT login` | The plugin bundles ChatGPT apps (`apps` in its manifest, or an `.app.json`), which Codex can't run on an API key. Leave the plugin out of Codex's cells. |
| `Codex reads the review plugin's .codex-plugin/plugin.json in place of its .claude-plugin/plugin.json, so reeval can't hold it to the parts it places: remove .codex-plugin/plugin.json from the plugin, or leave the plugin out of this cell` | Codex reads its own manifest (or a root `plugin.json` with an Agent Plugins `$schema`) first, and reeval reads only `.claude-plugin/plugin.json`. Remove the other manifest, or keep the plugin off Codex's cells. |
| `Claude Code loads the review plugin's MCP server "docs" itself, so reeval can't give it DOCS_TOKEN from the host: use only: ['skills'] and declare the server in mcp, where reeval fills it in` | Claude Code, Antigravity, Grok Build and Codex start a plugin's servers themselves. Leave `mcp` out of `only` and declare the server in the environment's `mcp`. |
| `Grok Build names the review plugin's MCP server "docs" by its own name, as it does mcp.docs, and keeps only one: rename one, or leave mcp out of the plugin with only` | Grok Build and Codex don't namespace a plugin's MCP servers. Rename one server, or leave `mcp` out of the plugin's `only`. |
| `Goose keys the review_docs and review:docs MCP servers both as review_docs, and keeps only one: rename one` | Goose keys an MCP server by its name lower-cased, without spaces, and with other punctuation as `_`, so a plugin's `review:docs` and `mcp.review_docs` are one server to it. Rename one. |
| `Cursor can't run the review plugin's Notification hooks (hooks/hooks.json): it runs SessionStart, SessionEnd, UserPromptSubmit, PreToolUse, PostToolUse, SubagentStop, PreCompact, Stop hooks; use only: ['skills', 'mcp'], or leave the plugin out of this cell` | The harness has no such event. Leave `hooks` out with `only`, or drop the event from the plugin. |
| `the audit plugin's subagent "reviewer" and the review plugin's would both be placed at .cursor/agents/reviewer.md: rename one, or narrow one with only` | Cursor takes plugin subagents and commands flat. Rename one, or leave `agents` (or `commands`) out of one plugin's `only`. |
| `pi names the review:docs MCP server review_docs, as it does the review_docs server, and keeps only one: rename one` | pi names a plugin's servers `<plugin>_<server>`. Rename the environment's server, or leave `mcp` out of the plugin's `only`. |
| `Antigravity reads the review plugin's plugin.json in place of its .claude-plugin/plugin.json, so reeval can't hold it to the parts it places: remove plugin.json from the plugin, or leave the plugin out of this cell` | agy installs a plugin with its own manifest (a root `plugin.json`, `.antigravity-plugin/`, `.gemini-plugin/` or `gemini-extension.json`) as a native one. Remove that manifest, or keep the plugin off Antigravity cells. |
| `the review plugin's rule "style.mdc" and the review plugin's rule "style.md" would both be placed at .cursor/rules/review-style.mdc: rename one, or narrow one with only` | Cursor unpacks each rule as `<plugin>-<name>.mdc`. Rename one rule file. |
| `Cursor can't run the review plugin's prompt hooks (hooks/hooks.json): it runs only command hooks; use only: ['skills', 'mcp'], or leave the plugin out of this cell` | Cursor and Antigravity run only `command` hooks. Leave `hooks` out with `only`, or make the hook a command. |
| `plugin.local('plugins/bare') has no .claude-plugin/plugin.json` | Point at the plugin's root, the directory with `.claude-plugin/plugin.json`. |
| `the review plugin has no hooks, so only: ['hooks'] matches nothing (it has skills, mcp)` | Name parts the plugin has in `only`. |
| `plugin "review" has a build, which runs in a Docker image layer: use docker()` | Builds run on Docker only. Run the eval on `docker()`, or leave `build` out on `fakeSandbox()`. |
| `the fetcher plugin's MCP server "search" runs npx, which fetches from registry.npmjs.org at start: add a build that installs it, or allow the host in egress` | Give the plugin a `build` that installs the server, or add the registry to `egress`. Under `proxy` and `allow-all` this is a note in the plan instead. |
| `the review plugin's manifest sets "skills", which reeval doesn't read: keep each part in its default place (skills/, agents/, commands/, hooks/hooks.json)` | Move the part to its default directory. |
| `environment review lists two plugins named "review", plugin.local('plugins/a') and plugin.local('plugins/b'): rename one, or leave one out` | One environment lists each plugin name once. A child environment's plugin replaces its parent's of the same name. |
| `the review plugin's MCP server "docs" would join mcp as review:docs, which mcp already names: rename mcp["review:docs"]` | Rename the environment's own server. |
| `the review plugin's MCP server "docs": "${HOME}" in .mcp.json names a variable; reeval fills in only ${CLAUDE_PLUGIN_ROOT} in a command and its args, …` | Pass the host variable through the server's `env` as a whole value, such as `"TOKEN": "${TOKEN}"`. |

A `plugin.git`, `plugin.npm` or `plugin.marketplace` plugin is fetched when an attempt starts, so the same problems with it fail its attempts as infra errors (exit 3), prefixed by `environment <name> (<eval>)`. An http plugin MCP server whose host a restricted egress policy doesn't allow fails the same way, as do a Cursor workspace file a plugin part would replace (`The workspace already has .cursor/agents/reviewer.md, which a plugin would replace: …`), a plugin Antigravity's CLI refuses to install (`agy plugin install <dir> exited 1: …`) and a marketplace that can't give the entry:

| Message after the prefix | Fix |
|---|---|
| `plugin.marketplace('lint@github:acme/tools'): github:acme/tools has no .claude-plugin/marketplace.json, so it isn't a Claude Code marketplace; name a plugin's own repository or package with plugin.git or plugin.npm` | Name a Claude Code marketplace, or the plugin's repository with `plugin.git`. |
| `plugin.marketplace('lnt@github:acme/tools'): the acme-tools marketplace (github:acme/tools) has no plugin "lnt" (did you mean "lint"?); it lists lint, format, review` | Name an entry the marketplace lists. |
| `the acme-tools marketplace's entry "lint" has source {"source":"hg","url":"…"}, which reeval doesn't fetch: it takes a ./path in the marketplace, or a github, url, git-subdir or npm source` | Name the plugin's own source with `plugin.git` or `plugin.npm`. |
| `the acme-tools marketplace's entry "lint" sets "commands", which reeval doesn't read: keep each part in the plugin's own directory` | Use a plugin whose parts are in their default places. |
| `the acme-tools marketplace's entry "lint" has ref "--upload-pack=…", which isn't a branch or tag` | The marketplace's entry is malformed: name the plugin's own source with `plugin.git`. `sha` must be a full 40-character commit SHA. |
| `the acme-tools marketplace's entry "lint" has a password or token in its url, which reeval would record: …` | Name the repository with `plugin.git` and fetch it with your own git credentials. |
| `the acme marketplace (https://git.example.com/acme/plugins.git) names its review plugin at file:///srv/review, a path on this machine: a remote marketplace may only name remote repositories or paths inside itself` | A remote marketplace can't reach this machine's files. Name the plugin with `plugin.git` or `plugin.local`, or use a local copy of the marketplace. |
| `the acme-tools marketplace's entry "lint" names npm package "git+ssh://…", which isn't a registry package name: reeval packs only registry packages from a marketplace` | Name the plugin's own source with `plugin.git` or `plugin.npm`. |

A plugin `build` runs in preflight, like an `install` layer: a failing command fails the attempts that use it with `Building the build of plugin.local('plugins/review') failed: build 1/2 exited 1: npm ci` (exit 3), and a `plugin.git` ref that moves between preflight and the attempt fails it with `The review plugin's ref moved during the run: …. Pin ref to a commit or tag.`

## During an attempt

| Message | Fix |
|---|---|
| `c.judge() has no turn to grade: call c.send() first` | Send a turn first, or pass `on`. |
| `c.judge() has no model. …` | Set `judge.model` in the config, group, eval or call. |
| `c.judge() state.checks names a command this attempt hasn't run: "npm tset"` | Name the command exactly as you passed it to `c.exec`, after running it. |
| `c.judge() on gives the output, but state.output is false` | Drop `output: false`, or drop `on`. |
| `expected file greet.js to exist` | The agent didn't create it. Check the transcript. |
| `expected command to succeed, exit code 1` | The command failed; the message has the tail of stderr and stdout. A check that fails on a transient (a dropped model stream in the code under test) is still a failed check, not an infra error: give its `c.exec` [`retries`](/reference/attempt-context#retrying-a-command). |
| `c.exec()'s retries must be a whole number, from 0 (got -1)` | Pass a whole number of extra tries. `interval` is 0 or more milliseconds. |
| `c.exec()'s retries: can't be used with background: true, whose exit reeval doesn't wait for: drop one of them (got 2)` | Retry a foreground command; a background one has no exit to retry on. |
| `expect(…).toSucceed() wasn't awaited. …` | Write `await expect(…)`. The eval fails even when the check passes. |
| `judge score 0.40 is below 0.8 for "…" (…)` | The judge said no. Read its input in the attempt's `judge-input.json`. |
| `Network is unreachable`, `connection refused` or `could not resolve host` in a check or the transcript, with nothing in the record's `egress.contacted` or `blocked` | Under the default, `'proxy'`, only HTTP(S) through the egress gateway leaves the sandbox, so raw TCP, UDP, SSH and database drivers get no connection. Set `egress: 'allow-all'` on the eval, or point the client at `HTTPS_PROXY`. The run's `proxy-only-egress` remark names the eval. See [Egress](/reference/egress#connections-that-never-reach-the-gateway). |
| `Eval timed out after 900s` | Raise `timeout`, or make the task smaller. |
| `out of memory: 1 process killed at the 4g limit (+4g swap); raise docker({ memory })` | A process in the sandbox hit its memory limit, so the attempt is an infra error, whatever its checks said. Raise `docker({ memory })`, or give Docker more memory. |
| `setup step 1/2 "npm ci" in environment base exited 1 after …` | Fix the setup step; its output is in the attempt's `setup.log`. An infra error. |
| `ANTHROPIC_API_KEY isn't set, so the Claude Code harness can't authenticate. …` | Set the key. |

For any failing attempt, `.reeval/debug` has its `debug.log` and `trace.ndjson`: see [Debug data](/reference/debug-data).

## Related

- [Debugging guide](/guide/debugging)
- [Debug data](/reference/debug-data)
- [reeval run](/reference/cli/run)
- [defineConfig](/reference/config)