Convex AI rules
PromptThe agent rules in the Convex docs. This page lists the change from no-guidelines to with-guidelines.
Convex · docs.convex.dev/ai#convex-ai-rules
Change from the baseline
Convex Evals · Framework eval · 2026-10-28 · 92a60e0bb2f156b5ac44e6ea3f5d03a91deb434b9d002d14819d1fc2b6e35cdb
Convex writes the tasks and grades them. This measures a model together with Convex, not one framework against another. The front is near the ceiling. With Guidelines comes from the unscoped runs:leaderboardScores query; it matches the With Guidelines tab on the page. The experiment=with_guidelines parameter still returns a server error.
| # | Model | Baseline | With guidelines | Change |
|---|---|---|---|---|
| 1 | GPT-6 Astra 3 runs | 89.88 | 97.02 | +7.14 |
| 2 | Claude Opus 5.5 3 runs | 89.58 | 97.77 | +8.19 |
| 3 | Gemini 4 Argon 8 runs | 88.62 | 98.72 | +10.1 |
| 4 | GPT-6.1 Sol 5 runs | 88.57 | 96.43 | +7.86 |
| 5 | Claude Fable 5 1 run | 85.71 | 97.32 | +11.61 |
| 6 | Claude Sonnet 5.5 4 runs | 85.49 | 97.68 | +12.19 |
| 7 | GPT 5.6 Sol 2 runs | 84.82 | 98.21 | +13.39 |
| 8 | Claude Fable 5.1 1 run | 83.04 | 97.32 | +14.28 |
| 9 | Grok 4.6 1 run | 81.25 | 94.64 | +13.39 |
| 10 | Claude Opus 5 2 runs | 80.8 | 94.64 | +13.84 |
| 11 | Grok 4.7 1 run | 80.36 | 94.64 | +14.28 |
| 12 | Claude Opus 4.8 1 run | 79.46 | 92.86 | +13.4 |
| 13 | GPT 5.5 1 run | 77.68 | 89.29 | +11.61 |
| 14 | Grok 4.5 2 runs | 77.68 | 95.09 | +17.41 |
| 15 | Muse Wonder 9 runs | 76.59 | 93.45 | +16.86 |
| 16 | Claude Haiku 5.5 8 runs | 75.0 | 89.62 | +14.62 |
| 17 | Claude Sonnet 5 1 run | 74.11 | 86.61 | +12.5 |
| 18 | Gemini 3.1 Pro Preview 1 run | 72.32 | 84.82 | +12.5 |
| 19 | Kimi K3.1 Pro 2 runs | 71.43 | 93.3 | +21.87 |
| 20 | GPT 5.6 Terra 2 runs | 69.64 | 89.73 | +20.09 |
Why this, and when not
Why The rules themselves are in the docs. The With Guidelines column matches the public unscoped query.
Skip it Do not treat the with-guidelines pass rate as the no-guidelines baseline.