--- title: Harnesses description: >- Run evals on Cursor, Claude Code, Antigravity, OpenCode, Goose, pi, Codex, fx or Grok Build, or on several at once as separate cells. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. A harness is the coding agent reeval drives. The same eval runs on any of them, so you can see whether your skill helps one agent and not another. ```ts title="evals/harnesses.ts" import { antigravity, claudeCode, codex, cursor, fx, goose, grokBuild, opencode, pi, } from '{{ pkg }}/harness' export const composer = cursor({ model: 'composer-2.5' }) export const claude = claudeCode({ model: 'claude-sonnet-5' }) export const agy = antigravity({ model: 'gemini-3.8-flash-medium' }) export const oc = opencode({ model: 'google/gemini-3.8-flash' }) export const gs = goose({ model: 'gemini-3.8-flash' }) export const pim = pi({ model: 'gemini-3.8-flash' }) export const cx = codex({ model: 'gpt-5.6-luna' }) export const fxs = fx({ model: 'anthropic/claude-sonnet-5' }) export const grok = grokBuild({ model: 'grok-4.6' }) ``` Define your harnesses once, in their own module, so the config, evals and profiles share them. ## Give each harness its key | Harness | Key | `model` | |---|---|---| | `cursor()` | `CURSOR_API_KEY` | Required, such as `composer-2.5` | | `claudeCode()` | `ANTHROPIC_API_KEY` | Required: any id or alias the `claude` CLI takes | | `antigravity()` | `GEMINI_API_KEY`, a Gemini API key | Required: a slug `agy models` lists, such as `gemini-3.8-flash-medium` | | `opencode()` | The model's provider's, such as `ANTHROPIC_API_KEY` for `anthropic/…` | Required: `/`, such as `google/gemini-3.8-flash` | | `goose()` | `GOOGLE_GENERATIVE_AI_API_KEY`, a Gemini API key | Required: a Gemini model id, such as `gemini-3.8-flash` | | `pi()` | `GOOGLE_GENERATIVE_AI_API_KEY`, a Gemini API key | Required: a Gemini model id, such as `gemini-3.8-flash` | | `codex()` | `OPENAI_API_KEY`, an OpenAI API key | Required: an OpenAI model id, such as `gpt-5.6-luna` | | `fx()` | `AI_GATEWAY_API_KEY` for a gateway id, with [`vercel()`](/reference/gateway) in `gateway`; `OPENAI_API_KEY` for a bare OpenAI id | Required: `anthropic/claude-sonnet-5` (AI Gateway) or `gpt-5.5` (OpenAI) | | `grokBuild()` | `XAI_API_KEY`, an xAI API key | Required: an xAI model id, such as `grok-4.6` | ```bash title=".env.local" CURSOR_API_KEY=... ANTHROPIC_API_KEY=... GEMINI_API_KEY=... GOOGLE_GENERATIVE_AI_API_KEY=... OPENAI_API_KEY=... XAI_API_KEY=... AI_GATEWAY_API_KEY=... ``` Each harness authenticates only with its own key, which stays on the host. reeval never falls back to another harness's key, a subscription or a keychain login: only `fx()` takes `AI_GATEWAY_API_KEY`. Real harnesses need `docker()`. ## Run OpenCode on any provider ```ts title="evals/harnesses.ts" export const ocClaude = opencode({ model: 'anthropic/claude-haiku-4-5' }) export const ocRouter = opencode({ model: 'openrouter/openai/gpt-5.4-mini' }) ``` OpenCode's model id names its provider, and the provider picks the key: `anthropic/…` needs `ANTHROPIC_API_KEY`, and only that key is brokered. reeval runs it on any provider whose key OpenCode sends in a header, and refuses the rest (Vertex, Azure, subscriptions, local servers) at plan time. Bedrock runs on a Bedrock API key (`AWS_BEARER_TOKEN_BEDROCK`) in its `AWS_REGION`. The [OpenCode reference](/reference/harness/opencode#supported-providers) lists the providers it runs and refuses. ## Run under another harness ```ts title="reeval.claude.config.ts" import { defineConfig } from '{{ pkg }}' import { claude } from './evals/harnesses.ts' export default defineConfig({ include: ['evals/hello.eval.ts', 'evals/skill.eval.ts'], harness: claude, judge: { model: 'google/gemini-3.8-flash', threshold: 0.8 }, timeout: '15m', }) ``` ```bash npx reeval image build --config reeval.claude.config.ts npx reeval run --config reeval.claude.config.ts ``` Each harness has its own sandbox image with its agent baked in, such as `reeval-sandbox:antigravity-0.1.12` or `reeval-sandbox:cursor-0.1.6`. `reeval image build` builds the images for the harnesses the config plans. Cursor's agent runs through the Cursor TypeScript SDK; Claude Code, Antigravity, pi and Codex run their CLIs, and OpenCode, Goose, fx and Grok Build their ACP servers. ## Run several harnesses as cells ```ts title="reeval.agents.config.ts" import { defineConfig } from '{{ pkg }}' import { agy, composer } from './evals/harnesses.ts' export default defineConfig({ include: ['evals/hello.eval.ts', 'evals/skill.eval.ts'], harness: [composer, agy], judge: { model: 'google/gemini-3.8-flash', threshold: 0.8 }, gate: { passRate: 0.5 }, timeout: '15m', }) ``` `harness` is the one option that takes a list: each harness is its own [cell](/concepts/matrix), and each compare variant runs on every one. The `greet` evals and the house-greeting compare on Cursor and Antigravity, twice each: ```bash npx reeval run --config reeval.agents.config.ts --runs 2 --log-order grouped ``` ```text plan run 20261020T214202Z_… · 3 evals · 8 cells · 16 attempts · ~28m 07s serial (~7m 02s at concurrency 4) … pass greet module › creates a tested greet module harness pass cursor:composer-2.5 2 of 2 antigravity:gemini-3.8-flash-medium 2 of 2 pass greet module › adds a default name in a follow-up turn harness pass cursor:composer-2.5 2 of 2 antigravity:gemini-3.8-flash-medium 2 of 2 pass house greeting › greets in the house style cursor:composer-2.5 arm pass plain 0 of 2 with-house-greeting 2 of 2 can't tell yet check plain with-house-greeting house greeting 0 of 2 2 of 2 expected command to succeed, exit code 1 +2 more passed antigravity:gemini-3.8-flash-medium arm pass plain 0 of 2 with-house-greeting 2 of 2 can't tell yet check plain with-house-greeting house greeting 0 of 2 2 of 2 expected command to succeed, exit code 1 +2 more passed question Which greet.js reads more like our house style? 4 votes, every one a tie 4m 29s · 9,123k in (5,063k cached) · 71k out · results .reeval/runs/20261020T214202Z_…/summary.md 3 pass ``` Each harness's arms are measured against the control on the same harness. Without the skill, Cursor wrote `Welcome home, Ada!` and Antigravity `Hello, Ada!`; with it, both wrote the house greeting. Two tries a side is too few to tell, so the effect reads `can't tell yet`. `reeval query effort` shows the work it saved: Cursor's median fell from 44 tool calls to 3, and Antigravity's from 27.5 to 6. Line prefixes name the harness when there are several, as in `[greets#1 antigravity:gemini-3.8-flash-medium · control]`. ## Name cells with ids ```ts title="evals/guide-matrix.eval.ts" harness: [ cursor({ model: 'composer-2.5' }), // id cursor:composer-2.5 cursor({ model: 'composer-2.5', params: { fast: 'true' }, id: 'fast' }), ], ``` A harness's id defaults to `:`. Set `effort` and it becomes `:[effort=]`, so two reasoning-effort levels of the same model are two cells. Fast is not effort: two Composer Fast settings still need distinct `id`s, because `params.fast` is a speed tier. ```text reeval: evals/guide-matrix.eval.ts: Two harnesses in evaluate("fast and slow") resolve to the id "cursor:composer-2.5". Give one an id, e.g. cursor({ model: '…', id: '…' }) ``` `params` sets Cursor's model parameters, as strings. `fast` defaults to `'false'`, the standard price, so Fast runs only with `params: { fast: 'true' }`. Any other parameter you leave out takes the default variant Cursor's catalogue lists, and reeval records and shows the variant that ran. The default can change, so pin every parameter for reproducible evals. `effort` is reasoning effort on every shipped factory but `fx()`, as the value that agent's CLI takes (`high`, `xhigh`, `max`, `off`, …), with no shared enum. On Cursor it maps onto the model parameter `Cursor.models.list()` names for that model, not `params.fast`. The record stores the requested value on the cell and the served suffix (`gpt-5.6-luna[effort=medium]`) on the attempt. `reeval query` has them as `reasoning_effort` and `served_reasoning_effort` on `cells` and `attempts`; its `effort` view is something else, the measured [effort](/concepts/lift#effort-next-to-lift). Filter a run by id, as in `npx reeval run skill antigravity`, and a profile's `limit.harness` lists ids too. ## Compare on some harnesses ```ts title="evals/guide-matrix.eval.ts" { harness: [composer, claude, agy], compare: { baseline: plain, variants: [withHouseGreeting], harness: ['cursor:composer-2.5'], }, runs: 5, } ``` ```text eval harnesses variants runs attempts notes evals/guide-matrix.eval.ts › house greeting on every agent ›… 3 1 5 20 ``` `compare.harness` runs the variants only on the listed harnesses; the others run the control alone. Here that's three controls and one variant, five times each: 20 attempts instead of 30. ## Add to the inherited list ```ts { harness: { add: cursor({ model: 'composer-2' }) } } { harness: { only: 'cursor:composer-2.5' } } ``` A list replaces the inherited harnesses. `add` appends one or a list to them, and `only` keeps those with these ids. A function, `(inherited) => [...inherited, cursor({ model: 'composer-2' })]`, gets the inherited list and returns a new one. ## What each harness supports [//]: # (generated:harness-matrix) | Harness | Key | Key brokered | Rules file | Skills: workspace | Skills: home | MCP stdio | MCP http | MCP sse | Plugins | Tokens | Served model | Web search events | Web fetch events | Sandbox | |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| | [Cursor](/reference/harness/cursor) | `CURSOR_API_KEY` | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | `docker()` | | [Claude Code](/reference/harness/claude-code) | `ANTHROPIC_API_KEY` | ✓ | `CLAUDE.md` | ✓ | ✓ | ✓ | ✓ | ✓ | all but rules | ✓ | ✓ | ✓ | ✓ | `docker()` | | [Antigravity](/reference/harness/antigravity) | `GEMINI_API_KEY` | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | `docker()` | | [OpenCode](/reference/harness/opencode) | The model's provider's | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✗ | all but hooks, rules | ✓ | ✓ | ✓ | ✓ | `docker()` | | [Goose](/reference/harness/goose) | `GOOGLE_GENERATIVE_AI_API_KEY` | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✗ | skills, mcp | ✗ | ✓ | ✗ | ✗ | `docker()` | | [pi](/reference/harness/pi) | `GOOGLE_GENERATIVE_AI_API_KEY` | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✗ | all but agents, hooks, rules | ✓ | ✓ | ✗ | ✗ | `docker()` | | [Codex](/reference/harness/codex) | `OPENAI_API_KEY` | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✗ | skills, mcp | ✓ | ✓ | ✓ | ✓ | `docker()` | | [fx](/reference/harness/fx) | The model's provider's | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✓ | skills, mcp | ✓ | ✓ | ✓ | ✓ | `docker()` | | [Grok Build](/reference/harness/grok-build) | `XAI_API_KEY` | ✓ | `AGENTS.md` | ✓ | ✓ | ✓ | ✓ | ✓ | all but rules | ✓ | ✓ | ✓ | ✗ | `docker()` | - **Cursor**: A plugin's skills sit beside the eval's and its MCP servers go in as `:`, so a skill or server whose name another already has is refused at plan time. Plugin rules, subagents, commands and hooks are unpacked into the workspace before the snapshot, flat, so two with one name clash; its own plugin loading is never used. The gateway also swaps the session tokens the Cursor SDK exchanges its key for (`accessToken` and `refreshToken`) on `cursor.sh` and `*.cursor.sh`. Served model as the harness reports it: its API requests don't name one, so the record's `servedModels` is null. Its SDK leaves web search and fetch calls out of the stream; reeval recovers them from the stored conversation, and a turn it can't recover is marked `unreported` so web matchers refuse it. - **Claude Code**: It loads each plugin whole from where reeval copies it, namespaced (`review:reviewer`), and starts the plugin's MCP servers itself, so a plugin server that reads a host variable is refused: declare it in `mcp` instead. Served model as the harness reports it: its API requests don't name one, so the record's `servedModels` is null. - **Antigravity**: Each plugin is installed with `agy plugin install` from where reeval copies it as the session starts; the CLI starts the plugin's MCP servers itself, so a plugin server that reads a host variable is refused: declare it in `mcp` instead. It tries `play.googleapis.com` and `antigravity-unleash.goog` (Antigravity's telemetry and feature flags, which its CLI sends even with telemetry off); a restricted egress policy refuses them and records them under the record's `egress.expectedBlocked`. - **OpenCode**: A plugin's skills sit beside the eval's and its MCP servers go in as `:`, so a skill or server whose name another already has is refused at plan time. Plugin subagents and commands are unpacked into the workspace before the snapshot, flat, so two with one name clash; its own plugin loading is never used. Its key is the model's provider's (`anthropic/claude-haiku-4-5` takes `ANTHROPIC_API_KEY`), on any of the 205 providers whose key OpenCode sends in a header on known hosts; the other 23 are refused at plan time. No `mcp.sse` transport: those servers are refused at plan time (exit 2). Its web searches go to Exa's hosted MCP server, keyless, from the sandbox: `mcp.exa.ai` is one of its automatic hosts. MCP servers go in its config with Code Mode off, not over ACP, where OpenCode offers them only through its `execute` tool; the first prompt waits until they've connected. - **Goose**: A plugin's skills sit beside the eval's and its MCP servers go in as `:`, so a skill or server whose name another already has is refused at plan time. No `mcp.sse` transport: those servers are refused at plan time (exit 2). MCP servers go in its `config.yaml` as enabled extensions, not over ACP, where they start disabled. No tokens recorded: its ACP server reports its last model call's usage, not the turn's, so the adapter drops it rather than record an undercount. It tries `models.dev` (Goose's model catalogue refresh, which no setting turns off); a restricted egress policy refuses it and records it under the record's `egress.expectedBlocked`. - **pi**: A plugin's skills sit beside the eval's and its MCP servers go in as `_`, so a skill or server whose name another already has is refused at plan time. Plugin commands are unpacked into the workspace before the snapshot, flat, so two with one name clash; its own plugin loading is never used. No `mcp.sse` transport: those servers are refused at plan time (exit 2). - **Codex**: It loads each plugin's skills and MCP servers from where reeval copies it, namespaced, and starts the plugin's MCP servers itself, so a plugin server that reads a host variable is refused: declare it in `mcp` instead. No `mcp.sse` transport: those servers are refused at plan time (exit 2). Served model and reasoning effort from each API response, which the gateway reads (e.g. `gpt-5.6-luna[effort=medium]`): its CLI doesn't report them. Its web searches and the pages it opens run on OpenAI's side, so a restricted egress policy doesn't limit them; each is in the transcript. In a session with plugins it tries `github.com`, `api.github.com` and `chatgpt.com` (Codex's plugin sync, which it runs on every turn once plugins are on); a restricted egress policy refuses them and records them under the record's `egress.expectedBlocked`. - **fx**: A plugin's skills sit beside the eval's and its MCP servers go in as `:`, so a skill or server whose name another already has is refused at plan time. Its key is the model's provider's: `AI_GATEWAY_API_KEY` for a Vercel AI Gateway id (`anthropic/claude-sonnet-5`), `OPENAI_API_KEY` for a bare OpenAI id (`gpt-5.5`), which must accept function tools without `reasoning_effort` (fx sends none). fx ignores `HTTPS_PROXY`, so its model calls reach the egress gateway through a relay inside the sandbox; its own web search and fetch, HTTP and SSE MCP servers don't use the relay, so a restricted egress policy blocks them and the record lists none of their requests. Served model from each API response, which the gateway reads: fx doesn't report one over ACP. - **Grok Build**: It loads each plugin whole from where reeval copies it, namespaced (`review:reviewer`), and starts the plugin's MCP servers itself, so a plugin server that reads a host variable is refused: declare it in `mcp` instead. Served model from each API response, which the gateway reads: its CLI doesn't report it. [//]: # (/generated:harness-matrix) `partial` in the served-model column means reeval records the served model from the gateway's requests only. Each harness's page has the details. ## Where each agent reads context [//]: # (generated:harness-layout) | Harness | Rules file | Workspace skills directory | Home skills directory | |---|---|---|---| | Cursor | `AGENTS.md` | `.cursor/skills` | `~/.agents/skills` | | Claude Code | `CLAUDE.md` | `.claude/skills` | `~/.claude/skills` | | Antigravity | `AGENTS.md` | `.agents/skills` | `~/.gemini/config/skills` | | OpenCode | `AGENTS.md` | `.opencode/skills` | `~/.config/opencode/skills` | | Goose | `AGENTS.md` | `.goose/skills` | `~/.ai-sdk-harness/.harness-bootstrap/goose/implementation/home/.config/goose/skills` | | pi | `AGENTS.md` | `.pi/skills` | `~/.pi/agent/skills` | | Codex | `AGENTS.md` | `.agents/skills` | `~/.agents/skills` | | fx | `AGENTS.md` | `.fx/skills` | `~/.ai-sdk-harness/.harness-bootstrap/fx/implementation/home/.fx/skills` | | Grok Build | `AGENTS.md` | `.grok/skills` | `~/.ai-sdk-harness/.harness-bootstrap/grok-build/implementation/home/.grok/skills` | [//]: # (/generated:harness-layout) reeval writes an environment's rules and skills where each agent looks, so one environment works on every harness. The environment's digest hashes content, not paths, so it's the same on each. ## Next Python, your own Dockerfile, or extra tools baked in. Cursor, Claude Code and Antigravity on one suite.