EQ-Bench Creative Writing v3
2026-10-29 · Docs and copy
- What it measures
- Creative writing on prompts: character and insight, interest and originality, prose quality, plot and character coherence, following the brief, world and atmosphere, plus avoiding cliches, show-off vocab, and metaphor overload.
- How
- EQ-Bench Creative Writing v3. Thirty-two writing prompts × 3 iterations (96 items), temp 0.7, min_p 0.1. Claude 4 Sonnet grades a rubric absolute score; Claude Sonnet 4.6 then runs pairwise matchups against neighbors (Glicko, with + margins) for relative Elo. Pairwise inputs are truncated to 4000 characters and judged both ways for position bias.
- What the score means
- The leaderboard metric is normalized Elo (higher is better). Relative ratings drift when models are added, so scores are anchored to DeepSeek-R1 = 1500 and ministral-3b = 200. Composite treats it as an arena board: twice the expected win rate vs the leader (leader = 100). Rubric, Slop, Repetition, and Length stay source-side auxiliaries and do not enter this site's composite.
- Caveats
- An LLM judge has self-preference and taste bias; the authors say this is not a roleplay eval and does not match your taste. English only. Source 🆕 marks newly added models, not this site's preliminary half-weight flag.
- Updates
- Source https://eqbench.com/creative_writing.html ; methodology https://eqbench.com/about.html#creative-writing-v3 ; harness https://github.com/EQ-bench/creative-writing-bench . This page fetched all 146 rows on 2026-10-29 from the CSV embedded in creative_writing.js (v=1.0.91), ordered by Elo descending.
Full board
146
| # | Model | Score |
|---|
| 1 | gpt-6-astra | 2176.9 |
| 2 | kimi-k3.1-pro | 2172.6 |
| 3 | claude-fable-5-1 | 2155.3 |
| 4 | claude-opus-5 | 2135.0 |
| 5 | gpt-6-sol | 2126.0 |
| 6 | kimi-k3 | 2083.3 |
| 7 | GLM-5.3 | 2076.4 |
| 8 | claude-opus-5-5 | 2050.9 |
| 9 | grok-4.7 | 2012.2 |
| 10 | muse-wonder | 2007.0 |
| 11 | Oox-alpha | 1973.7 |
| 12 | gpt-5.6-sol | 1973.5 |
| 13 | claude-fable-5 | 1943.6 |
| 14 | Aaion-3.5 | 1930.6 |
| 15 | muse-spark-1.1 | 1928.2 |
| 16 | Cclaude-opus-4-7 | 1914.9 |
| 17 | AAltworld/Hemmingway-1 | 1907.4 |
| 18 | muse-spark-1.3 | 1906.9 |
| 19 | gpt-5.6-terra | 1856.2 |
| 20 | gpt-5.5 | 1844.5 |