Benchmarks

OpenCharts publishes no scores of its own for the ranked models. It reads what publishers other than the models' makers publish (Epoch AI's Benchmarking Hub and the Arena leaderboard dataset, both Creative Commons Attribution), keeps the best published run per model and test, measures every score against the best published result on that test, and averages. Every number on the site links back to the publisher that produced it and says who produced it, and the snapshot date is printed on every page. Every model also carries its lab's country of origin.

The hub reads 31 benchmarks across 8 categories: Reasoning, Knowledge, Coding, Math, Agentic, Multimodal, Human preference, Long context. Only categories with at least 2 comparable tests are averaged into the Index; a category that is a single benchmark is shown beside it.

1. The best published run per model and benchmark counts.
2. Each score is measured against the frontier on its test: the best result the publisher lists across its whole file, whether or not that model is tracked here. Percent scores become the share of that frontier above random guessing, so the leader reads 100 and sitting a harder exam never lowers a model. Arena ratings become the expected win rate against the board leader (100 points behind reads 72). Open-ended values sit on a log scale above a fixed floor. A test with fewer than 3 models scored is shown but not counted.
3. Scores are averaged inside each category. A category is part of the Index once the snapshot has at least 2 comparable tests in it, and a model's mean counts once it has results on at least half of those tests.
4. The OpenCharts Index is the mean of the counting category means, so a lab cannot climb by publishing many tests of one kind.
5. A model is ranked once it has at least 6 comparable tests across 3 counting categories; otherwise it is provisional. The grade reads the Index as a distance from the frontier: A+ at 95+, A at 90+, A- at 85+, B+ at 80+, B at 75+, B- at 70+, C+ at 65+, C at 60+, C- at 50+, D at 40+, F at 0+. Within 5% of the best published results, on average.
6. Every Index carries a leave-one-out band (where it lands if any single test is dropped) and every rank carries the places that band covers. Head-to-head comparisons call any gap under 2 points a tie, because one item on a 45-problem exam is 2.2 points.

Category leaders are the third-party published, ranked models with the best qualifying mean in each category; leaders of single-test categories are marked as sitting beside the Index. The full method, with every threshold, is on the methodology page.

Third-party published means a publisher other than the model's maker produced every counted score. It is not the same claim as “independently re-run”, so every score also carries one of three provenance labels, in Epoch AI's own classification of its hub: Epoch-run (Epoch AI ran the evaluation itself, under settings it documents per model, with a public log), Leaderboard (the benchmark's official leaderboard or a third-party evaluator), and Self-reported (the model's maker published the number in a model card or launch post and Epoch AI collected it). One benchmark is developer-reported throughout (Cybench); their cards and pages say so in amber.

Internal, independent verification TBD means no publisher has scored the engine yet, and the scores shown are OpenCharts' own runs on the same public tests: each card shows the dataset version, item count, repeats, grader and run date. Internal scores rank in the same table but never lead a spotlight, and the label stays until a publisher verifies the engine. Tests OpenCharts has not run for such an engine read TBD rather than a dash.

The hard-set score is the mean of a model's relative scores on the 8 hardest, most general evaluations in the set (Humanity's Last Exam, ARC-AGI-2, SWE-bench Verified, FrontierMath (Tiers 1–3), FrontierMath Tier 4, Terminal-Bench, GDPval, METR Time Horizon), shown once a model has published results on at least 4 of them and counting third-party scores only. A newly released model usually has fewer; it is listed as waiting on more results, with the mean it has so far, and enters the ranking as publishers add results.

• The best published run counts, so labs that publish many effort levels get the benefit of their best one.
• Agent benchmarks compare model-plus-harness systems; harness choice can move a score by ten points.
• Some benchmarks are authored or graded by a lab that also competes on them. Every such test page says so.
• Publishers' confidence intervals are not propagated; the leave-one-out band is a stand-in.
• The frontier moves whenever a better result is published, so Index values compare within a snapshot, not across snapshots.

The whole ranking is published from the same data the pages render:

• /benchmarks/llms.txt as plain text, in the llms.txt style AI assistants read.
• /benchmarks/rankings.json as JSON with every model, rank band, category mean, score, frontier, source and provenance.

Credit the publishers as their licenses require and OpenCharts for the Index; the attribution lines are on the methodology page.