Why this benchmark is useful
Editorial brief pendingWe publish the methodology and ledger first; benchmark-specific analysis ships after desk review.
Benchmarks / Performance
AA time to first token
TTFT
Median seconds until the first token returns on the model's default API provider with a ~1k-token input prompt.
What this does not measure
- End-to-end task time for agent loops — TTFT is one hop, not a full coding session.
- Non-default providers — AA pins one default route per model for comparability.
Coverage map
- Task family
- Performance
- Format
- Single-shot API request with streaming enabled; median P50 over 72 hours.
- Scoring
- Lower seconds is faster.
- Maintainer
- Artificial Analysis
How to read the scores
Reading guide pending. Use task format, scoring method, and source dates in the ledger until the desk brief ships.
What it does not cover
- End-to-end task time for agent loops — TTFT is one hop, not a full coding session.
- Non-default providers — AA pins one default route per model for comparability.
Scores
69 rowsEvidence ledger
6969 rows
129.43sbest score
69source-checked
1sources
2026-07-21source date
Where the rows land
0255075100
Normalized to this benchmark's axis (0–100). Open the table for raw units.
Newest receipts
- GPT 5.4129.43s ·
- GPT-5.6 Terra100.45s ·
- GPT-5.297.23s ·
- Claude Sonnet 588.49s ·
- GPT-5.6 Luna83.80s ·
Scores use this benchmark's own unit and axis, not a universal quality score.
| # | Model | Release date | Score | Provenance | Trust |
|---|---|---|---|---|---|
| 1 | GPT 5.4OpenAI | 2026-03-05 | 129.43s | 2026-07-21 | Source-checked |
| 2 | GPT-5.6 TerraOpenAI | 2026-07-09 | 100.45s | 2026-07-21 | Source-checked |
| 3 | GPT-5.2OpenAI | 2025-12-11 | 97.23s | 2026-07-21 | Source-checked |
| 4 | Claude Sonnet 5Anthropic | 2026-06-30 | 88.49s | 2026-07-21 | Source-checked |
| 5 | GPT-5.6 LunaOpenAI | 2026-07-09 | 83.80s | 2026-07-21 | Source-checked |
| 6 | GPT-5.6 SolOpenAI | 2026-07-09 | 76.91s | 2026-07-21 | Source-checked |
| 7 | Claude Fable 5Anthropic | 2026-06-09 | 68.33s | 2026-07-21 | Source-checked |
| 8 | Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)Anthropic | 2026-02-17 | 57.98s | 2026-07-21 | Source-checked |
| 9 | GPT-5.3 Codex (xhigh)OpenAI | 2026-02-05 | 41.56s | 2026-07-21 | Source-checked |
| 10 | GPT-5.1OpenAI | 2025-11-13 | 38.96s | 2026-07-21 | Source-checked |
| 11 | GPT 5.5OpenAI | 2026-04-23 | 31.38s | 2026-07-21 | Source-checked |
| 12 | Grok 4.3xAI | 2026-04-30 | 31.32s | 2026-07-21 | Source-checked |
| 13 | Claude Opus 4.8Anthropic | 2026-05-28 | 30.13s | 2026-07-21 | Source-checked |
| 14 | Gemini 3.1 Pro PreviewGoogle | 2026-02-19 | 21.91s | 2026-07-21 | Source-checked |
| 15 | Grok 4.20 ReasoningxAI | 2026-03-05 | 16.81s | 2026-07-21 | Source-checked |
| 16 | Claude Opus 4.7Anthropic | 2026-04-16 | 16.72s | 2026-07-21 | Source-checked |
| 17 | Claude Opus 4.6 (Adaptive Reasoning, Max Effort)Anthropic | 2026-02-05 | 14.02s | 2026-07-21 | Source-checked |
| 18 | GPT-5.5 (high)OpenAI | 2026-04-23 | 13.95s | 2026-07-21 | Source-checked |
| 19 | Gemini 3.5 FlashGoogle | 2026-05-19 | 12.11s | 2026-07-21 | Source-checked |
| 20 | Grok 4.5xAI | 2026-07-08 | 9.69s | 2026-07-21 | Source-checked |
| 21 | Gemini 3.5 Flash (medium)Google | 2026-05-19 | 8.89s | 2026-07-21 | Source-checked |
| 22 | GPT-5.4 MiniOpenAI | 2026-03-17 | 8.20s | 2026-07-21 | Source-checked |
| 23 | GPT-5.5 (medium)OpenAI | 2026-04-23 | 6.88s | 2026-07-21 | Source-checked |
| 24 | Kimi K3Moonshot | 2026-07-16 | 6.19s | 2026-07-21 | Source-checked |
| 25 | Gemini 3.1 Flash LiteGoogle | 2026-05-07 | 4.90s | 2026-07-21 | Source-checked |
| 26 | OpenAI o3OpenAI | 2025-04-16 | 4.84s | 2026-07-21 | Source-checked |
| 27 | GPT-5.4 NanoOpenAI | 2026-03-17 | 3.46s | 2026-07-21 | Source-checked |
| 28 | MiMo-V2.5-ProXiaomi | 2026-04-22 | 2.38s | 2026-07-21 | Source-checked |
| 29 | Claude Opus 4.6Anthropic | 2026-02-05 | 1.85s | 2026-07-21 | Source-checked |
| 30 | LFM2.5-8B-A1BLiquid | 2026-06-08 | 1.79s | 2026-07-21 | Source-checked |
| 31 | Qwen 3.7 PlusAlibaba | 2026-06-02 | 1.69s | 2026-07-21 | Source-checked |
| 32 | GPT-5.5 (low)OpenAI | 2026-04-23 | 1.61s | 2026-07-21 | Source-checked |
| 33 | Claude Opus 4.7 (Non-reasoning, High Effort)Anthropic | 2026-04-16 | 1.55s | 2026-07-21 | Source-checked |
| 34 | Qwen3.7 MaxAlibaba | 2026-05-19 | 1.55s | 2026-07-21 | Source-checked |
| 35 | Gemma 4 12B (Non-reasoning)Google | 2026-06-10 | 1.49s | 2026-07-21 | Source-checked |
| 36 | Gemma 4 12B (Reasoning)Google | 2026-06-07 | 1.46s | 2026-07-21 | Source-checked |
| 37 | Claude Opus 4.5Anthropic | 2025-11-24 | 1.32s | 2026-07-21 | Source-checked |
| 38 | MiniMax M3MiniMax | 2026-05-31 | 1.26s | 2026-07-21 | Source-checked |
| 39 | Claude Sonnet 4.6Anthropic | 2026-02-17 | 1.18s | 2026-07-21 | Source-checked |
| 40 | Kimi K2.7 CodeMoonshot | 2026-06-12 | 1.17s | 2026-07-21 | Source-checked |
| 41 | MiniMax M2.7MiniMax | 2026-04-15 | 1.17s | 2026-07-21 | Source-checked |
| 42 | Kimi K2.6Moonshot | 2026-04-20 | 1.08s | 2026-07-21 | Source-checked |
| 43 | Kimi K2.5Moonshot | 2026-01-27 | 1.02s | 2026-07-21 | Source-checked |
| 44 | MiniMax M2.5MiniMax | 2026-04-01 | 1.02s | 2026-07-21 | Source-checked |
| 45 | DeepSeek V4 Pro (Reasoning, Max Effort)DeepSeek | 2026-04-24 | 1.01s | 2026-07-21 | Source-checked |
| 46 | Gemma 4 31B ITGoogle | 2026-03-01 | 0.98s | 2026-07-21 | Source-checked |
| 47 | Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIA | 2026-06-04 | 0.88s | 2026-07-21 | Source-checked |
| 48 | GLM-5.1Zhipu | 2026-04-07 | 0.86s | 2026-07-21 | Source-checked |
| 49 | GLM-5.2Zhipu | 2026-06-13 | 0.84s | 2026-07-21 | Source-checked |
| 50 | Muse Spark 1.1Meta | 2026-07-09 | 0.83s | 2026-07-21 | Source-checked |
| 51 | Gemini 3 Flash PreviewGoogle | 2025-12-17 | 0.72s | 2026-07-21 | Source-checked |
| 52 | Step 3.7 FlashStepFun | 2026-06-01 | 0.72s | 2026-07-21 | Source-checked |
| 53 | HyperNova 60B 2605Multiverse | 2026-05-06 | 0.60s | 2026-07-21 | Source-checked |
| 54 | Llama 4 MaverickMeta | 2026-04-05 | 0.60s | 2026-07-21 | Source-checked |
| 55 | Mistral Large 3Mistral | 2025-12-02 | 0.59s | 2026-07-21 | Source-checked |
| 56 | Llama 4 ScoutMeta | 2026-04-05 | 0.56s | 2026-07-21 | Source-checked |
| 57 | Command ACohere | 2026-03-15 | 0.36s | 2026-07-21 | Source-checked |
| 58 | North Mini CodeCohere | 2026-06-09 | 0.23s | 2026-07-21 | Source-checked |
| 59 | Claude 3 OpusAnthropic | 2024-03-04 | 0.00s | 2026-07-21 | Source-checked |
| 60 | DeepSeek Coder V2DeepSeek | 2024-05-15 | 0.00s | 2026-07-21 | Source-checked |
| 61 | DeepSeek R1DeepSeek | 2025-01-20 | 0.00s | 2026-07-21 | Source-checked |
| 62 | Gemini 1.0 UltraGoogle | 2024-02-08 | 0.00s | 2026-07-21 | Source-checked |
| 63 | Gemini 3 ProGoogle | 2025-11-18 | 0.00s | 2026-07-21 | Source-checked |
| 64 | GPT-5.4 ProOpenAI | 2026-03-05 | 0.00s | 2026-07-21 | Source-checked |
| 65 | GPT-5.5 ProOpenAI | 2026-04-23 | 0.00s | 2026-07-21 | Source-checked |
| 66 | LongCat-2.0Meituan | 2026-06-29 | 0.00s | 2026-07-21 | Source-checked |
| 67 | MiniCPM5-1B (Reasoning)OpenBMB | 2026-06-04 | 0.00s | 2026-07-21 | Source-checked |
| 68 | Muse SparkOther | 2026-01-01 | 0.00s | 2026-07-21 | Source-checked |
| 69 | OpenAI o1OpenAI | 2024-09-12 | 0.00s | 2026-07-21 | Source-checked |
What it covers
Data-quality note pending. Every ledger row still carries source URL, source date, and ingest timestamp.
Where it breaks down
No ceiling note is recorded yet. Treat clustering near the top as a warning that the benchmark may no longer separate frontier models.
Tools that report it
Receipts
Sources and further reading
AA performance methodology· checked 2026-06-10
The Pack · Editorial newsletter
New cards in your inbox. Free.
One short email when a card ships or changes status. No tracking, no third-party analytics. Unsubscribe in one click.