--- title: Compare agents description: >- Run one suite on Cursor and Antigravity at once and read pass rate, lift and effort per agent. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. **Goal:** see how different coding agents do on the same evals, side by side, including whether a skill helps each of them. A [harness](/guide/harnesses) list makes each agent its own [cell](/concepts/matrix). The evals don't change: every agent gets the same prompt, a fresh sandbox of its own and the same checks. ## The files ```ts title="evals/harnesses.ts" import { antigravity, claudeCode, cursor } from '{{ pkg }}/harness' export const composer = cursor({ model: 'composer-2.5' }) export const claude = claudeCode({ model: 'claude-sonnet-5' }) export const agy = antigravity({ model: 'gemini-3.8-flash-medium' }) ``` ```ts title="reeval.agents.config.ts" import { defineConfig } from '{{ pkg }}' import { docker } from '{{ pkg }}/sandbox' import { agy, composer } from './evals/harnesses.ts' export default defineConfig({ include: ['evals/hello.eval.ts', 'evals/skill.eval.ts'], harness: [composer, agy], judge: { model: 'google/gemini-3.8-flash', threshold: 0.8 }, gate: { passRate: 0.5 }, timeout: '15m', }) ``` The evals are a greeting eval and a house-greeting skill compare. The second is like the compare from [Prove a skill helps](/recipes/skill-lift), so each agent gets a control and a variant. Every harness needs a `model`. Cursor's `fast` parameter defaults to `'false'`, the standard price, so the record shows the served model `composer-2.5[fast=false]`. Each harness has an id, a key and an image of its own: | Harness | Id (in cell keys and filters) | Key in `.env.local` | Sandbox image | |---|---|---|---| | `cursor({ model: 'composer-2.5' })` | `cursor:composer-2.5` | `CURSOR_API_KEY` | `reeval-sandbox:cursor-0.1.6` | | `claudeCode({ model: 'claude-sonnet-5' })` | `claude-code:claude-sonnet-5` | `ANTHROPIC_API_KEY` | `reeval-sandbox:claude-code-1.0.160-patch.b17cde50` | | `antigravity({ model: 'gemini-3.8-flash-medium' })` | `antigravity:gemini-3.8-flash-medium` | `GEMINI_API_KEY` | `reeval-sandbox:antigravity-0.1.12` | Each key stays on the host and is swapped in by the egress gateway only on that agent's API endpoints (see [Credential brokering](/concepts/credential-brokering)). ## Build the images and check the plan ```bash npx reeval image build --dry-run --config reeval.agents.config.ts ``` ```text images 2 sandbox images (nothing built: --dry-run) docker · cursor · 2 evals base reeval-sandbox:cursor-0.1.0 (rebuild: the harness bootstrap) docker · antigravity · 2 evals base reeval-sandbox:antigravity-0.1.12 (rebuild: the harness bootstrap) ``` ```bash npx reeval run --dry-run --config reeval.agents.config.ts --runs 2 ``` ```text plan 3 evals · 8 cells · 16 attempts · ~12m 31s serial (~3m 08s at concurrency 4) eval harnesses variants runs attempts notes evals/hello.eval.ts › greet module › creates a tested greet module 2 — 2 4 evals/hello.eval.ts › greet module › adds a default name in a… 2 — 2 4 evals/skill.eval.ts › house greeting › greets in the house style 2 1 2 8 ``` A filter that names a harness runs only its cells, and reeval asks only for that harness's key: ```bash npx reeval run antigravity --config reeval.agents.config.ts --runs 2 ``` ```text plan 2 evals · 2 cells · 4 attempts · ~12m 00s serial (~3m 00s at concurrency 4) reeval: Missing credentials: GEMINI_API_KEY. Set them in the environment or .env.local. ``` ## Run both agents ```bash npx reeval run --config reeval.agents.config.ts --runs 2 ``` A real run on Cursor and Antigravity: ```text plan run 20261020T214202Z_… · 3 evals · 8 cells · 16 attempts · ~28m 07s serial (~7m 02s at concurrency 4) … pass greet module › creates a tested greet module harness pass cursor:composer-2.5 2 of 2 antigravity:gemini-3.8-flash-medium 2 of 2 pass greet module › adds a default name in a follow-up turn harness pass cursor:composer-2.5 2 of 2 antigravity:gemini-3.8-flash-medium 2 of 2 pass house greeting › greets in the house style cursor:composer-2.5 arm pass plain 0 of 2 with-house-greeting 2 of 2 can't tell yet check plain with-house-greeting house greeting 0 of 2 2 of 2 expected command to succeed, exit code 1 +2 more passed antigravity:gemini-3.8-flash-medium arm pass plain 0 of 2 with-house-greeting 2 of 2 can't tell yet check plain with-house-greeting house greeting 0 of 2 2 of 2 expected command to succeed, exit code 1 +2 more passed question Which greet.js reads more like our house style? 4 votes, every one a tie 4m 29s · 9,123k in (5,063k cached) · 71k out · results .reeval/runs/20261020T214202Z_…/summary.md 3 pass ``` What it says: - **Both agents pass `hello.eval.ts`**, so that eval can't tell them apart: a row per harness, `2 of 2` each. - **The skill gets both to pass.** Without it, Cursor's controls wrote `Welcome home, Ada!` and `Welcome to our house, Ada!`, and Antigravity's `Hello, Ada!`. The `house greeting` check is the one that tells the arms apart. - **`can't tell yet` is the honest verdict at two tries a side.** Run `--runs 4` or more when the effect, or the effort, is what you're measuring; `-v` adds each arm's difference and interval. - **Effort differs more than pass rate.** `reeval query effort` has Cursor's median going from 44 tool calls to 3 with the skill, and Antigravity's from 27.5 to 6. - **Each harness's arms are against the control on the same harness.** Cell keys are `//`. A harness is an agent and a model together. Cursor with composer against Antigravity with Gemini can't tell you whether the agent or the model made the difference; compare two models on one harness for that. ## Ask across runs `reeval query factor_effects` sets one factor against another, within a run (`controlled`) and pooled over comparable runs (`pooled`). From the same history: ```bash npx reeval query "SELECT eval_key, factor, tier, points, lo, hi, pass_verdict, effort_verdict FROM factor_effects" ``` ```text eval_key factor tier points lo hi pass_verdict effort_verdict … evals/skill.eval.ts › house greeting › greets in the house… environment pooled 1 0.307 1 effect fewer · harness pooled 0 -0.436 0.436 noise insufficient ``` Pooled over comparable runs, the skill has an effect on both pass rate and tool calls, and the choice of agent has none that the data can show. [`reeval analyze`](/reference/cli/analyze) reads the same facts and explains them: ```text - Across all evals, comparing the antigravity harness against cursor yielded a pooled pass-rate lift of +0 pts (95% CI −44…+44, verdict: noise) and effort verdict: insufficient (tools p = 0.3333 across 3 strata). [F12] … verify: reeval run --runs 6 --tag skill ``` The token line comes from the harnesses' own reports; `reeval query` has them per attempt in `input_tokens`, `cache_read_tokens`, `output_tokens` and `served_model`. reeval prices no models. There's no Claude Code run on this page: the Anthropic account behind these captures had no credit, so its turns ended in `error` (`Credit balance is too low` in the transcript). ## Variations | Variation | How | |---|---| | Two settings of one model | `cursor({ model: 'composer-2.5', params: { fast: 'true' }, id: 'cursor:composer-2.5-fast' })` next to the standard one, at several times the price. Give same-kind harnesses distinct ids | | Add an agent to one group | `{ harness: (inherited) => [...inherited, claude] }` extends the list instead of replacing it | | Keep PRs cheap | A `pr` profile with `limit: { harness: ['cursor:composer-2.5'] }` runs one agent on PRs and all of them nightly (see [Nightly matrix and Pages](/recipes/nightly-pages)) | | Ask which agent did better | A question with `across: 'harness'` judges the agents' diffs against each other after the run, per arm and run, as reeval's nightly `questions` check does on Cursor and pi. Its win rates annotate; see [Questions](/guide/questions) | | Rate limits | `antigravity({ model, maxConcurrent: 2 })` caps that agent's attempts at once | | A bigger suite | The [dogfood case study](/recipes/case-study-dogfood) ran two skills on three harder evals | ## Next Each agent's options, rules and skill locations. Lift, effort and factor effects across runs.