Furniture Assembly
60 photos of IKEA furniture partway through assembly, some with a deliberate mistake. Given the photo and the official manual, the model must say whether the build is right so far and, if not, name the step that went wrong and describe it. Epoch AI runs it.
Understanding photos, diagrams and documents alongside text. One of 4 comparable tests in multimodal, a category the Index averages.
Read with care. A language model grades each description of a mistake, leniently; the step itself is checked against an answer key. Volunteers who had never built these products averaged 63%.
Publisher: Epoch AI (internal runs)What a model is asked to do
Look at a photo of furniture partway through assembly, compare it with the manual, and say whether it has been built correctly so far.
For exampleThe photo shows a dresser frame after step 11. One side panel is on back to front, so the drawer rails face outward. Name the step where it went wrong.
Why it matters. Matching a diagram to a real object is the skill behind repairing an appliance or checking an installation from a photo.
The example is original and illustrative, not an item from the dataset.
- Models scored here
- 37
- Who produced the numbers
- Run by Epoch AI
- Items graded
- n = 60 (1.7% each)
- Best published result
- 90.0%
- Newest release covered
- November 9, 2026 (active)
- Chance level
- 30%
- License
- CC BY 4.0 (via Epoch AI)
- Direction
- Higher is better
- Scale
- Published as a share of items solved. For the Index, each score is measured as a share of the best published result (90.0%) above chance: random guessing (30%) reads 0 and the frontier reads 100, so a high-guessing exam is not flattered.
- Provenance
- Epoch AI ran the evaluation itself, under settings it documents per model, and publishes the log of every answer.
Snapshot November 10, 2026
Ranking on this test
37 models on Furniture Assembly
Best published run per model, and how close each one comes to the best published result on this test (the frontier reads 100, measured above chance). Every number links to the publisher that produced it. Internal runs are marked and carry their run details on the model page. One item on this test is 1.7 points, so read gaps smaller than that as noise.
The best models, ranked here, working inside Theo.
29 of the models on this page run inside Theo today. Theo picks the right one for each step and always shows which engine answered.