Why this benchmark is useful
Editorial brief pendingWe publish the methodology and ledger first; benchmark-specific analysis ships after desk review.
Benchmarks / Performance
AA output speed
Output tokens per second
Median output tokens per second after the first chunk arrives, measured on each model's default API provider with a ~1k-token input prompt (AA medium workload).
What this does not measure
- IDE or chat-app latency — only first-party API routes AA tracks as the default provider.
- Quality or reasoning depth — speed-only slice. A fast model can still fail hard tasks.
Coverage map
- Task family
- Performance
- Format
- Streaming generation after first token; median over the past 72 hours on AA's harness.
- Scoring
- Higher tokens/s is faster. AA uses OpenAI tiktoken o200k_base for token counts.
- Maintainer
- Artificial Analysis
How to read the scores
Reading guide pending. Use task format, scoring method, and source dates in the ledger until the desk brief ships.
What it does not cover
- IDE or chat-app latency — only first-party API routes AA tracks as the default provider.
- Quality or reasoning depth — speed-only slice. A fast model can still fail hard tasks.
Scores
69 rowsEvidence ledger
6969 rows
401 t/sbest score
69source-checked
1sources
2026-07-21source date
Where the rows land
0255075100
Normalized to this benchmark's axis (0–100). Open the table for raw units.
Newest receipts
- Step 3.7 Flash401 t/s ·
- HyperNova 60B 2605381 t/s ·
- LFM2.5-8B-A1B343 t/s ·
- Gemini 3.1 Flash Lite315 t/s ·
- Gemini 3.5 Flash (medium)299 t/s ·
Scores use this benchmark's own unit and axis, not a universal quality score.
| # | Model | Release date | Score | Provenance | Trust |
|---|---|---|---|---|---|
| 1 | Step 3.7 FlashStepFun | 2026-06-01 | 401 t/s | 2026-07-21 | Source-checked |
| 2 | HyperNova 60B 2605Multiverse | 2026-05-06 | 381 t/s | 2026-07-21 | Source-checked |
| 3 | LFM2.5-8B-A1BLiquid | 2026-06-08 | 343 t/s | 2026-07-21 | Source-checked |
| 4 | Gemini 3.1 Flash LiteGoogle | 2026-05-07 | 315 t/s | 2026-07-21 | Source-checked |
| 5 | Gemini 3.5 Flash (medium)Google | 2026-05-19 | 299 t/s | 2026-07-21 | Source-checked |
| 6 | Gemini 3.5 FlashGoogle | 2026-05-19 | 287 t/s | 2026-07-21 | Source-checked |
| 7 | Grok 4.20 ReasoningxAI | 2026-03-05 | 227 t/s | 2026-07-21 | Source-checked |
| 8 | Gemini 3 Flash PreviewGoogle | 2025-12-17 | 218 t/s | 2026-07-21 | Source-checked |
| 9 | Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIA | 2026-06-04 | 215 t/s | 2026-07-21 | Source-checked |
| 10 | Qwen3.7 MaxAlibaba | 2026-05-19 | 207 t/s | 2026-07-21 | Source-checked |
| 11 | GPT-5.6 LunaOpenAI | 2026-07-09 | 203 t/s | 2026-07-21 | Source-checked |
| 12 | GLM-5.2Zhipu | 2026-06-13 | 196 t/s | 2026-07-21 | Source-checked |
| 13 | GPT-5.4 MiniOpenAI | 2026-03-17 | 180 t/s | 2026-07-21 | Source-checked |
| 14 | GPT-5.4 NanoOpenAI | 2026-03-17 | 163 t/s | 2026-07-21 | Source-checked |
| 15 | GPT-5.6 TerraOpenAI | 2026-07-09 | 157 t/s | 2026-07-21 | Source-checked |
| 16 | OpenAI o3OpenAI | 2025-04-16 | 155 t/s | 2026-07-21 | Source-checked |
| 17 | GPT 5.4OpenAI | 2026-03-05 | 152 t/s | 2026-07-21 | Source-checked |
| 18 | Gemini 3.1 Pro PreviewGoogle | 2026-02-19 | 136 t/s | 2026-07-21 | Source-checked |
| 19 | Gemma 4 12B (Reasoning)Google | 2026-06-07 | 129 t/s | 2026-07-21 | Source-checked |
| 20 | GPT-5.3 Codex (xhigh)OpenAI | 2026-02-05 | 127 t/s | 2026-07-21 | Source-checked |
| 21 | Grok 4.3xAI | 2026-04-30 | 124 t/s | 2026-07-21 | Source-checked |
| 22 | Muse Spark 1.1Meta | 2026-07-09 | 122 t/s | 2026-07-21 | Source-checked |
| 23 | Gemma 4 12B (Non-reasoning)Google | 2026-06-10 | 119 t/s | 2026-07-21 | Source-checked |
| 24 | Llama 4 MaverickMeta | 2026-04-05 | 110 t/s | 2026-07-21 | Source-checked |
| 25 | GPT-5.1OpenAI | 2025-11-13 | 110 t/s | 2026-07-21 | Source-checked |
| 26 | MiniMax M3MiniMax | 2026-05-31 | 97 t/s | 2026-07-21 | Source-checked |
| 27 | North Mini CodeCohere | 2026-06-09 | 96 t/s | 2026-07-21 | Source-checked |
| 28 | GPT 5.5OpenAI | 2026-04-23 | 90 t/s | 2026-07-21 | Source-checked |
| 29 | Claude Sonnet 5Anthropic | 2026-06-30 | 88 t/s | 2026-07-21 | Source-checked |
| 30 | GPT-5.2OpenAI | 2025-12-11 | 87 t/s | 2026-07-21 | Source-checked |
| 31 | MiniMax M2.5MiniMax | 2026-04-01 | 87 t/s | 2026-07-21 | Source-checked |
| 32 | GLM-5.1Zhipu | 2026-04-07 | 85 t/s | 2026-07-21 | Source-checked |
| 33 | Grok 4.5xAI | 2026-07-08 | 77 t/s | 2026-07-21 | Source-checked |
| 34 | GPT-5.5 (high)OpenAI | 2026-04-23 | 77 t/s | 2026-07-21 | Source-checked |
| 35 | Llama 4 ScoutMeta | 2026-04-05 | 76 t/s | 2026-07-21 | Source-checked |
| 36 | GPT-5.5 (medium)OpenAI | 2026-04-23 | 76 t/s | 2026-07-21 | Source-checked |
| 37 | GPT-5.5 (low)OpenAI | 2026-04-23 | 76 t/s | 2026-07-21 | Source-checked |
| 38 | DeepSeek V4 Pro (Reasoning, Max Effort)DeepSeek | 2026-04-24 | 73 t/s | 2026-07-21 | Source-checked |
| 39 | Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)Anthropic | 2026-02-17 | 72 t/s | 2026-07-21 | Source-checked |
| 40 | Claude Fable 5Anthropic | 2026-06-09 | 70 t/s | 2026-07-21 | Source-checked |
| 41 | Claude Opus 4.5Anthropic | 2025-11-24 | 68 t/s | 2026-07-21 | Source-checked |
| 42 | GPT-5.6 SolOpenAI | 2026-07-09 | 67 t/s | 2026-07-21 | Source-checked |
| 43 | Claude Opus 4.6 (Adaptive Reasoning, Max Effort)Anthropic | 2026-02-05 | 64 t/s | 2026-07-21 | Source-checked |
| 44 | Mistral Large 3Mistral | 2025-12-02 | 64 t/s | 2026-07-21 | Source-checked |
| 45 | Claude Opus 4.8Anthropic | 2026-05-28 | 63 t/s | 2026-07-21 | Source-checked |
| 46 | Claude Opus 4.7Anthropic | 2026-04-16 | 61 t/s | 2026-07-21 | Source-checked |
| 47 | Command ACohere | 2026-03-15 | 61 t/s | 2026-07-21 | Source-checked |
| 48 | Claude Sonnet 4.6Anthropic | 2026-02-17 | 60 t/s | 2026-07-21 | Source-checked |
| 49 | MiniMax M2.7MiniMax | 2026-04-15 | 59 t/s | 2026-07-21 | Source-checked |
| 50 | MiMo-V2.5-ProXiaomi | 2026-04-22 | 58 t/s | 2026-07-21 | Source-checked |
| 51 | Claude Opus 4.6Anthropic | 2026-02-05 | 57 t/s | 2026-07-21 | Source-checked |
| 52 | Kimi K2.6Moonshot | 2026-04-20 | 56 t/s | 2026-07-21 | Source-checked |
| 53 | Qwen 3.7 PlusAlibaba | 2026-06-02 | 53 t/s | 2026-07-21 | Source-checked |
| 54 | Claude Opus 4.7 (Non-reasoning, High Effort)Anthropic | 2026-04-16 | 50 t/s | 2026-07-21 | Source-checked |
| 55 | Kimi K2.5Moonshot | 2026-01-27 | 50 t/s | 2026-07-21 | Source-checked |
| 56 | Kimi K2.7 CodeMoonshot | 2026-06-12 | 47 t/s | 2026-07-21 | Source-checked |
| 57 | Kimi K3Moonshot | 2026-07-16 | 38 t/s | 2026-07-21 | Source-checked |
| 58 | Gemma 4 31B ITGoogle | 2026-03-01 | 36 t/s | 2026-07-21 | Source-checked |
| 59 | Claude 3 OpusAnthropic | 2024-03-04 | 0 t/s | 2026-07-21 | Source-checked |
| 60 | DeepSeek Coder V2DeepSeek | 2024-05-15 | 0 t/s | 2026-07-21 | Source-checked |
| 61 | DeepSeek R1DeepSeek | 2025-01-20 | 0 t/s | 2026-07-21 | Source-checked |
| 62 | Gemini 1.0 UltraGoogle | 2024-02-08 | 0 t/s | 2026-07-21 | Source-checked |
| 63 | Gemini 3 ProGoogle | 2025-11-18 | 0 t/s | 2026-07-21 | Source-checked |
| 64 | GPT-5.4 ProOpenAI | 2026-03-05 | 0 t/s | 2026-07-21 | Source-checked |
| 65 | GPT-5.5 ProOpenAI | 2026-04-23 | 0 t/s | 2026-07-21 | Source-checked |
| 66 | LongCat-2.0Meituan | 2026-06-29 | 0 t/s | 2026-07-21 | Source-checked |
| 67 | MiniCPM5-1B (Reasoning)OpenBMB | 2026-06-04 | 0 t/s | 2026-07-21 | Source-checked |
| 68 | Muse SparkOther | 2026-01-01 | 0 t/s | 2026-07-21 | Source-checked |
| 69 | OpenAI o1OpenAI | 2024-09-12 | 0 t/s | 2026-07-21 | Source-checked |
What it covers
Data-quality note pending. Every ledger row still carries source URL, source date, and ingest timestamp.
Where it breaks down
No ceiling note is recorded yet. Treat clustering near the top as a warning that the benchmark may no longer separate frontier models.
Receipts
Sources and further reading
AA performance methodology· checked 2026-06-10
The Pack · Editorial newsletter
New cards in your inbox. Free.
One short email when a card ships or changes status. No tracking, no third-party analytics. Unsubscribe in one click.