Convex
LibraryA backend with queries and mutations. The official default rank is no guidelines and no web access.
Apache-2.0 · www.convex.dev · github.com/get-convex/convex
Framework eval
Convex Evals · Framework eval · 2026-10-28 · 92a60e0bb2f156b5ac44e6ea3f5d03a91deb434b9d002d14819d1fc2b6e35cdb
Convex writes the tasks and grades them. This measures a model together with Convex, not one framework against another. The front is near the ceiling. With Guidelines comes from the unscoped runs:leaderboardScores query; it matches the With Guidelines tab on the page. The experiment=with_guidelines parameter still returns a server error.
| # | Model | Baseline | With guidelines | No guidelines + web | Cost per run |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra 3 runs | 89.88 | 97.02 | 91.07 | $6.05 |
| 2 | Claude Opus 5.5 3 runs | 89.58 | 97.77 | 95.09 | $3.63 |
| 3 | Gemini 4 Argon 8 runs | 88.62 | 98.72 | 95.76 | $1.62 |
| 4 | GPT-6.1 Sol 5 runs | 88.57 | 96.43 | 92.19 | $1.19 |
| 5 | Claude Fable 5 1 run | 85.71 | 97.32 | 89.29 | $7.90 |
| 6 | Claude Sonnet 5.5 4 runs | 85.49 | 97.68 | 88.39 | $1.45 |
| 7 | GPT 5.6 Sol 2 runs | 84.82 | 98.21 | 89.29 | $2.52 |
| 8 | Claude Fable 5.1 1 run | 83.04 | 97.32 | 89.29 | $8.63 |
| 9 | Grok 4.6 1 run | 81.25 | 94.64 | — | $2.97 |
| 10 | Claude Opus 5 2 runs | 80.8 | 94.64 | 85.71 | $4.46 |
| 11 | Grok 4.7 1 run | 80.36 | 94.64 | — | $4.65 |
| 12 | Claude Opus 4.8 1 run | 79.46 | 92.86 | 85.71 | $4.47 |
| 13 | GPT 5.5 1 run | 77.68 | 89.29 | — | $6.69 |
| 14 | Grok 4.5 2 runs | 77.68 | 95.09 | 80.36 | $1.58 |
| 15 | Muse Wonder 9 runs | 76.59 | 93.45 | 84.46 | $6.49 |
| 16 | Claude Haiku 5.5 8 runs | 75.0 | 89.62 | 78.44 | $0.15517 |
| 17 | Claude Sonnet 5 1 run | 74.11 | 86.61 | 80.36 | $1.86 |
| 18 | Gemini 3.1 Pro Preview 1 run | 72.32 | 84.82 | — | $3.99 |
| 19 | Kimi K3.1 Pro 2 runs | 71.43 | 93.3 | — | $2.05 |
| 20 | GPT 5.6 Terra 2 runs | 69.64 | 89.73 | 79.46 | $1.93 |
Install and use
npm i convex
Why this, and when not
Why The 112-task no-guidelines board stands on its own. With-guidelines and web are other columns and are not merged into the rank.
Skip it If you already use Supabase or your own database, do not switch to add two official boards together.