All benchmarks
AgenticHard set

FrontierSWE v2

34 open-ended engineering and research projects, such as rebuilding a version-control system in another language or training a weather-forecasting model. Each gets a 20-hour budget and a deterministic verifier that gives partial credit. Mean of five trials per task.

Long-horizon tasks with tools: terminals, desktops, browsers, whole jobs. One of 6 comparable tests in agentic, a category the Index averages.

Read with care. Run by its creator, Proximal, in Proximal's own agent harness. No third party has reproduced the results, and Proximal says the tasks and scoring may still change.

Publisher: Proximal

What a model is asked to do

Build a whole piece of software or a research result alone, with a 20-hour budget and an automated checker waiting at the end.

For exampleRebuild a spreadsheet's formula engine from its documentation alone, in a language of your choice, until it gives the same answers as the original on thousands of saved workbooks.

Why it matters. This is a working day of unattended engineering, not a ticket. It shows whether a model can hold a plan together for hours and still finish.

The example is original and illustrative, not an item from the dataset.

Models scored here
22
Who produced the numbers
Official leaderboard
Items graded
n = 34 (2.9% each)
Best published result
69.9%
Newest release covered
November 9, 2026 (active)
License
CC BY 4.0 (via Epoch AI)
Direction
Higher is better
Scale
Published as a share of items solved. For the Index, each score is a share of the best published result (69.9%), so the frontier reads 100. Guessing earns nothing on this test, so no chance correction applies.
Provenance
Produced by the benchmark's own leaderboard or a third-party evaluator, not by the model's maker.

Snapshot November 10, 2026

Ranking on this test

22 models on FrontierSWE v2

Best published run per model, and how close each one comes to the best published result on this test (the frontier reads 100). Every number links to the publisher that produced it. Internal runs are marked and carry their run details on the model page. One item on this test is 2.9 points, so read gaps smaller than that as noise.

#ModelScoreProduced by
1Claude Fable 5.2Anthropic69.9%Leaderboard
2GPT-6 AstraOpenAI65.5%Leaderboard
3Claude Opus 5.5Anthropic62.3%Leaderboard
4Claude Sonnet 5.5Anthropic61.9%Leaderboard
5Gemini 4 CarbonGoogle61.2%Leaderboard
6Claude Fable 5.1Anthropic56.3%Leaderboard
7Gemini 4 ArgonGoogle55.0%Leaderboard
8Claude Opus 5Anthropic52.0%Leaderboard
9Claude Fable 5Anthropic47.0%Leaderboard
10Claude Haiku 5.5Anthropic43.8%Leaderboard
11GPT-5.6 SolOpenAI32.2%Leaderboard
12GLM 5.3Zhipu AI30.2%Leaderboard
13Grok 4.7xAI29.5%Leaderboard
14Kimi K3Moonshot AI25.9%Leaderboard
15Muse Spark 1.3Meta25.8%Leaderboard
16Grok 4.6xAI25.3%Leaderboard
17Gemini 3.7 FlashGoogle20.3%Leaderboard
18Gemini 3.8 FlashGoogle19.6%Leaderboard
19GLM 5.3 FlashZhipu AI18.1%Leaderboard
20Qwen3.8 MaxAlibaba17.8%Leaderboard
21Muse Spark 1.2Meta12.0%Leaderboard
22InklingThinking Machines4.1%Leaderboard
Theo

The best models, ranked here, working inside Theo.

29 of the models on this page run inside Theo today. Theo picks the right one for each step and always shows which engine answered.

Open Theo Chatllms.txt and JSON for agents