--- title: Python, Flutter and more description: >- Give the agent the toolchain your task needs, from an official image, extra packages or your own Dockerfile. --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.reeval.ai/llms.txt > Use this file to discover all available pages before exploring further. **Goal:** run evals whose tasks need more than Node: Python, a CLI tool, a seeded database or a heavy SDK such as Flutter. What's baked into the image belongs to the [sandbox](/guide/sandboxes), `docker()`. reeval adds what the agent needs on top of it: Node, git and the harness's CLI. | You need | Use | Example | |---|---|---| | A language's official image | `docker({ from })` | Python | | A few packages on reeval's image | `docker({ install })` | `jq` | | Files, a database, several steps | `docker({ dockerfile })` starting `FROM ${REEVAL_BASE}` | SQLite | | A large SDK image | `docker({ from })` with a longer timeout | Flutter | ## Python from the official image ```ts title="evals/python.eval.ts" import { defineEnvironment, evaluate, expect } from '{{ pkg }}' import { docker } from '{{ pkg }}/sandbox' const python = defineEnvironment('python', { sandbox: docker({ from: 'python:3.12-slim', id: 'python' }), }) // reeval bakes Node (for the harness bridge) and the Cursor CLI onto // the official Python image the first time this runs, then reuses it. evaluate( 'writes and tests fizzbuzz in python', { environment: python }, async (c) => { await c.send( 'Write fizzbuzz.py with a function fizzbuzz(n) returning the usual ' + 'FizzBuzz string for n, and test_fizzbuzz.py with unittest tests ' + 'for 3, 5, 15 and 7. Standard library only.', ) await expect(c.exec('python3 --version')).toSucceed() await expect(c.exec('python3 -m unittest -q')).toSucceed() }, ) ``` `from` takes any Debian or Ubuntu based image. `id: 'python'` names the sandbox in cell keys (`cursor:composer-2.5/-/python`). ## Packages on reeval's image ```ts title="evals/jq.eval.ts" import { defineEnvironment, evaluate, expect } from '{{ pkg }}' import { docker } from '{{ pkg }}/sandbox' const withJq = defineEnvironment('jq', { sandbox: docker({ install: [ 'apt-get update && apt-get install -y --no-install-recommends jq ' + '&& rm -rf /var/lib/apt/lists/*', ], }), }) evaluate( 'summarises a JSON file with jq', { environment: withJq }, async (c) => { await c.send( 'Write orders.json with five orders (id, customer, total), then ' + 'count.sh, a bash script that prints the number of orders over ' + '100 using jq. Make it executable.', ) await expect(c.exec('jq --version')).toSucceed() await expect(c.exec('./count.sh')).toSucceed() }, ) ``` `install` commands run once, as root with `HOME=/home/reeval`, with no secrets and no workspace. The agent itself runs as the unprivileged `reeval` user, so bake packages in here rather than having it `apt-get` them. reeval caches the result by the base image and the exact command list, so any edit builds a new layer. ## Your own Dockerfile ```dockerfile title="docker/sqlite/Dockerfile" ARG REEVAL_BASE FROM ${REEVAL_BASE} RUN apt-get update \ && apt-get install -y --no-install-recommends sqlite3 \ && rm -rf /var/lib/apt/lists/* COPY seed.sql /opt/seed.sql ``` ```ts title="evals/sqlite.eval.ts" import { defineEnvironment, evaluate, expect } from '{{ pkg }}' import { docker } from '{{ pkg }}/sandbox' const sqlite = defineEnvironment('sqlite', { sandbox: docker({ dockerfile: '../docker/sqlite/Dockerfile' }), }) evaluate( 'queries a seeded SQLite database', { environment: sqlite }, async (c) => { await c.send( 'Load /opt/seed.sql into app.db with sqlite3, then write report.sql ' + 'that lists customer names alphabetically, one per line.', ) const report = await c.exec('sqlite3 app.db < report.sql') await expect(report).toSucceed() expect(report.stdout).toBe('Ada\nGrace\nLinus\n') }, ) ``` reeval passes `--build-arg REEVAL_BASE=`, so `FROM ${REEVAL_BASE}` layers on an image that already has the agent's CLI. The path is relative to the eval file, and the build context is the Dockerfile's directory. ## Build and run ```bash npx reeval image build --dry-run ``` ```text images 4 sandbox images (nothing built: --dry-run) docker · cursor · 15 evals base reeval-sandbox:cursor-0.1.0 (rebuild: the harness bootstrap) docker+0ab0e263 · cursor + install · 1 eval base reeval-sandbox:cursor-0.1.0 (rebuild: the harness bootstrap) install 1 command reeval-install: (build unless cached) python · cursor · 1 eval from python:3.12-slim reeval-sandbox:cursor-0.1.0-python-3-12-slim-1d044754 (rebuild: Node, git and the harness bootstrap on top) docker:sqlite · cursor · 1 eval dockerfile docker/sqlite/Dockerfile reeval-dockerfile:sqlite-9630558fd3724574 (docker build (layer cache), tagged by content hash) harness reeval-dockerfile:sqlite-9630558fd3724574 (inherited: FROM ${REEVAL_BASE}, which is reeval-sandbox:cursor-0.1.0) ``` ```bash npx reeval image build npx reeval run python sqlite jq --output-logs full ``` A real run on Cursor: ```text plan run 20261020T214032Z_… · 3 evals · 3 cells · 3 attempts · ~34s serial (~11s at concurrency 4) … reeval 0.10.0 · run 20261020T214032Z_… · 3 eval files · harness cursor:composer-2.5 · sandbox docker · concurrency 4 · grouped logs … [writes#1 cursor] ▶ evals/python.eval.ts › writes and tests fizzbuzz in python [writes#1 cursor] +0:00 sandbox ready 2s [writes#1 cursor] +0:02 turn 1 done 6s · 1 tool call, 2 file edits [writes#1 cursor] +0:08 $ python3 --version exit 0 65ms [writes#1 cursor] +0:08 $ python3 -m unittest -q exit 0 231ms … pass summarises a JSON file with jq pass writes and tests fizzbuzz in python pass queries a seeded SQLite database 37s · 167k in (79k cached) · 1,653 out · results .reeval/runs/20261020T214032Z_…/summary.md 3 pass Set analysis.model in reeval.config.ts for answers to what each eval asks. ``` Without `reeval image build`, `reeval run` builds `from`, `dockerfile` and `install` images on first use, one at a time, and the other attempts wait for them. ## A heavy SDK: Flutter Large toolchain images work the same way. This one was checked with `--dry-run` only: ```ts title="evals/flutter.eval.ts" import { evaluate, expect } from '{{ pkg }}' import { docker } from '{{ pkg }}/sandbox' const flutter = docker({ from: 'ghcr.io/cirruslabs/flutter:3.38.10', id: 'flutter', }) evaluate( 'adds a counter widget test', { sandbox: flutter, timeout: '20m' }, async (c) => { await c.send( 'Create a Flutter app called counter with flutter create, then ' + 'add a widget test that taps the + button twice and expects 2.', ) await expect(c.exec('cd counter && flutter test')).toSucceed() }, ) ``` ```text images 1 sandbox image (nothing built: --dry-run) flutter · cursor · 1 eval from ghcr.io/cirruslabs/flutter:3.38.10 reeval-sandbox:cursor-0.1.0-ghcr-io-cirruslabs-flutter-3-38-10-ef09bba3 (rebuild: Node, git and the harness bootstrap on top) ``` - Pin the tag. A moving tag (`stable`) changes the toolchain between runs. - Image builds and setup steps don't count towards `timeout`, but `flutter create` and the first `flutter test` inside the turn do, so give the eval room. - An idle agent container takes about 300 MB, and an SDK's builds take far more. Lower `--concurrency` on small machines. Keep the image's user as `root` and leave `/reeval` alone: reeval's bootstrap and the harness live there. `install` doesn't combine with `dockerfile`; put those commands in the Dockerfile as `RUN` lines. ## Variations | Variation | How | |---|---| | An image that already has the harness | `docker({ image: 'ghcr.io/acme/agent:1' })` is used as is | | Per-attempt steps with secrets | `npm ci` or a database migration belongs in an environment's `setup`, not in `install`, which never sees secrets or the workspace | | Other harnesses | A `FROM ${REEVAL_BASE}` Dockerfile builds once per harness, on that harness's image (see [Compare agents](/recipes/compare-agents)) | | Registries only | Add `egress: { allow: [...DEFAULT_EGRESS_ALLOW] }` so the agent can install packages but reach nothing else (see [Offline evals](/recipes/offline)) | ## Next Every option, image tag and cache rule. Limit where the sandbox can connect.