---
title: Compare with past runs
description: Prevent regressions by comparing the outcomes of two recorded runs.
---
> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

`reeval compare` diffs two runs one evaluation at a time and names each one that regressed, was fixed, or can't be compared. It reads the runs you recorded to the history folder ([History](/evaluations/runs/recording)). For more information about the baseline and variants inside one run, see [Measure lift against a baseline](/results/compare/lift).

## Compare two runs

With no arguments, `reeval compare` takes the most recent run as the head and picks the base for you: the newest recorded run at a commit in the head's Git ancestry. When no run is in the ancestry, the base is the previous complete run. Run it after a run finishes:

```bash
npx reeval compare
```

To choose both runs, name the base first and the head second:

```bash
npx reeval compare latest~1 latest
```

A run is `latest`, `latest~N`, a run id or a unique prefix of one, or a Git SHA prefix. `reeval history` lists them.

When no earlier recorded run exists, `reeval compare` says so and lists every evaluation as `new`.

## Read the diff

`reeval compare` prints one block per evaluation that changed, in file order, and collapses each run of unchanged evaluations to `+N same`. A regressed block lists the assertions whose outcomes changed, and a block that can't be compared names what changed:

```text
can't compare    adds a tested multiply function
  changed          harness config
  base             20261024T145533Z_…  pass
  head             20261024T145534Z_…  fail
  tests pass       failed
    expected command to succeed, exit code 1
  +1 more passed

+1 same

regressed        adds a LangGraph node
  base             20261020T230147Z_958e309_cursor-agent_aamj
  head             20261020T230148Z_958e309_cursor-agent_qghb
  graph compiles   pass → fail
    expected command to succeed, exit code 1
  state schema     not run
  +1 more passed
```

The `can't compare` block comes from [`with-history`](https://github.com/invertase/reeval/tree/main/examples/with-history), whose scripted agent changed between the runs. A harness that fails with nothing changed reads `regressed`.

Each block and the tally use these words:

| Word | Meaning |
| :--- | :--- |
| `regressed` | The evaluation passed in the base and fails in the head. |
| `fixed` | The evaluation failed in the base and passes in the head. |
| `same` | The outcome is the same in both runs. |
| `can't compare` | The evaluation's source files, environment, harness config, image, agent CLI, judge, agent context, or gateway changed between the runs, so a changed outcome has another explanation. The block names what changed. |
| `infra error` | One side ended in an infrastructure error, so it has no outcome to compare. |
| `new` | Only the head has the evaluation. |
| `removed` | Only the base has the evaluation. |

When two runs differ only in which cells ran, `reeval compare` compares the cells both have and names the cells only one has. When the runs share no cell, or a shared cell changed, the evaluation reads `can't compare`. It also prints `can't compare` for an outcome change when one side's only failures are attempts an interrupt canceled, or when the served judge model changed. To print every evaluation with each cell's lift, tokens, tool calls, and time from base to head, and the associated stack changes, add `-v`.

## Fail on a regression

To make the diff a gate, add `--fail-on-regression`:

```bash
npx reeval compare --fail-on-regression
```

`reeval compare` exits with code 1 when an evaluation `regressed`, or when a `new` or `can't compare` evaluation fails, since no earlier run provides a passing result for comparison. Otherwise it exits with code 0, whatever else changed. Add `--md` to write the diff into a job summary. For more information about the GitHub Actions job, see [CI](/evaluations/runs/ci).

## Reference

For more information about every flag, see [`reeval compare`](/reference/cli#reeval-compare).