EBR-bench
Ten playthroughs of Earthborne Rangers, a little-known card-based board game, through a text interface: eight to practice and take notes, two that count. The score is the share of 21 objectives, which the model has to discover for itself, reached in the better of the two scored games. Epoch AI runs it five times per model.
Long-horizon tasks with tools: terminals, desktops, browsers, whole jobs. One of 6 comparable tests in agentic, a category the Index averages.
Read with care. An outside audit of the first version (Generality Labs) found it well built as a game test but a limited measure of learning from experience. Older models were scored under earlier settings and may not be directly comparable.
Publisher: Epoch AI (internal runs)What a model is asked to do
Play a little-known card-based adventure game ten times through a text interface, keeping notes, and score on the last two games.
For exampleNobody tells you what earns points. In the practice games you work out that helping a traveler and reaching the ridge both count, write that down, and plan the scored games around it.
Why it matters. Getting better at an unfamiliar job by doing it is what people do in their first weeks at work. This checks whether practice helps a model at all.
The example is original and illustrative, not an item from the dataset.
- Models scored here
- 30
- Who produced the numbers
- Run by Epoch AI
- Items graded
- n = 21 (4.8% each)
- Best published result
- 90.5%
- Newest release covered
- November 9, 2026 (active)
- License
- CC BY 4.0 (via Epoch AI)
- Direction
- Higher is better
- Scale
- Published as a share of items solved. For the Index, each score is a share of the best published result (90.5%), so the frontier reads 100. Guessing earns nothing on this test, so no chance correction applies.
- Provenance
- Epoch AI ran the evaluation itself, under settings it documents per model, and publishes the log of every answer.
Snapshot November 10, 2026
Ranking on this test
30 models on EBR-bench
Best published run per model, and how close each one comes to the best published result on this test (the frontier reads 100). Every number links to the publisher that produced it. Internal runs are marked and carry their run details on the model page. One item on this test is 4.8 points, so read gaps smaller than that as noise.
The best models, ranked here, working inside Theo.
29 of the models on this page run inside Theo today. Theo picks the right one for each step and always shows which engine answered.