Evals
- ScoringEach task is scored from its untouched baseline at 0% to its ceiling at 100%.
- AttemptsEvery model makes 8 attempts per task, with web search over a copy of the web frozen at October 1, 2026.
- Held-out testsIn Drawing to CAD, for example, the agent builds its pipeline on 30 public example parts, and the frozen pipeline is then scored on hidden parts in a separate sandbox. How grading works
Leaderboard
| # | Model | Score with 95% interval | |||
|---|---|---|---|---|---|
Whiskers and ± are 95% intervals across tasks. Cost is model usage at list prices and time is the agent's working time, both per attempt: one model working on one task, once. Select a model for its score by domain. How scoring works
Elo ratings
A second leaderboard from the same results. Every pair of models meets on every task: the higher score wins, and scores within the task's regrade tolerance tie. A Bradley–Terry fit puts each model on an Elo scale, with the untouched task baseline fixed at 1000.
| # | Model | Elo | Elo with 95% interval |
|---|---|---|---|
Showing All domains, set by the domain filter above. Bars run from the baseline at 1000, and whiskers are 95% intervals from refitting on resampled tasks.