---
title: Size runs with profiles
description: Size a run for pull requests, nightly runs, and local work.
---
> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

Size a [run](/reference/glossary#run) with a named preset in the `reeval.config.ts` file: what evaluations get by default, caps over every evaluation, a budget, and which tags run. For the definition, see [Profile](/reference/glossary#profile). One set of evaluation files then serves a cheap pull-request run and a thorough nightly one. For more information about wiring a profile into a workflow, see [CI](/evaluations/runs/ci).

## Define a profile

In the `reeval.config.ts` file, add a `profiles` map with one entry per kind of run:

```ts title="reeval.config.ts"
import { defineConfig } from '{{ pkg }}'

export default defineConfig({
  defaultProfile: { ci: 'pr' },
  profiles: {
    pr: { limit: { runs: 1 }, budget: { attempts: 10 } },
    nightly: {
      defaults: { runs: 4 },
      record: 'always',
      budget: { duration: '2h' },
    },
  },
})
```

For a runnable project, see [`with-profiles`](https://github.com/invertase/reeval/tree/main/examples/with-profiles). For one profile per harness, each in Docker with its own judge, see the examples' shared [`profiles.ts`](https://github.com/invertase/reeval/blob/main/examples/profiles.ts).

An evaluation that sets its own `runs` keeps it over the profile's `defaults`. For more information about the order, see [The cascade](/reference/options#the-cascade).

To pick a profile, pass `--profile`:

```bash
npx reeval run --profile nightly
```

The relevant start of the plan line names the profile and its selection source:

```text
plan  profile nightly (--profile) · 5 evals · 13 cells · 35 attempts · …
```

The `REEVAL_PROFILE` environment variable names a profile the same way. `--profile` takes precedence when both are set.

## Pick a default profile

The `defaultProfile` option chooses a profile when neither `--profile` nor `REEVAL_PROFILE` names one. In the `reeval.config.ts` file, make pull-request runs the default in CI:

```ts title="reeval.config.ts"
  defaultProfile: { ci: 'pr' },
```

reeval picks `defaultProfile.ci` when the `CI` variable is set and `defaultProfile.local` when it is not. Without a match, the run has no profile. The plan line reports the choice as `profile pr (auto: CI is set)`. reeval exits with code 2 and lists the profiles in the `reeval.config.ts` file when no profile defines the requested name.

## Limit a profile

A profile's `limit` narrows what evaluations ask for. In the `reeval.config.ts` file, restrict the `pr` profile to one harness:

```ts title="reeval.config.ts"
    pr: {
      limit: {
        runs: 1,
        compare: false,
        harness: ['claude-code:claude-sonnet-5'],
      },
      budget: { attempts: 10 },
    },
```

The plan notes show each cut:

```text
runs 4 → 1 (profile pr)
compare off (profile pr)
skipped: no harness left: profile pr allows claude-code:claude-sonnet-5
```

For more information about each `limit` key, see [`limit`](/reference/profiles#limit).

## Pin an evaluation

An evaluation or group's `pin` option names the keys a profile's `limit` may not narrow: `'runs'`, `'compare'`, or `'harness'`. For a regression that must keep its variant on every run, pin `compare` and `runs` in the `evals/brand.eval.ts` file:

```ts title="evals/brand.eval.ts"
pin: ['compare', 'runs'],
```

Under the `pr` profile, the plan lists both cells and notes `pinned: compare, runs`. A group's `pin` covers every evaluation in it.

## Set a budget

A profile's `budget` caps the plan and the run. Preview a run against its budget:

```bash
npx reeval run --profile pr --dry-run
```

The relevant end of the plan line shows whether the plan fits:

```text
… · 6 attempts · budget 5 attempts ✗
```

A run over the cap stops before it starts:

```text
reeval: this run plans 6 attempts, over budget.attempts 5 (profile pr).
Narrow it with --tag or --runs, or raise the budget.
(--force runs it anyway)
```

For more information about `attempts` and `duration`, see [`budget`](/reference/profiles#budget).

## Reference

For more information about every profile key and the keys `limit` narrows, see [Profiles reference](/reference/profiles). For more information about the `pin` option, see [Options](/reference/options#pin).