Why this benchmark is useful
Editorial brief pendingWe publish the methodology and ledger first; benchmark-specific analysis ships after desk review.
Benchmarks / Coding
LiveCodeBench
Contamination-resistant coding problems drawn from recent contest releases — AA reruns models on the public harness.
CodingSolidActiveLow contamination riskSince 2024
What this does not measure
- Multi-file refactors or IDE agent loops.
- Private codebase context — problems are public contest slices only.
Coverage map
- Task family
- Coding
- Format
- Timed coding tasks with public grader.
- Scoring
- pass@1 percentage from AA runs.
- Maintainer
- LiveCodeBench authors
How to read the scores
Reading guide pending. Use task format, scoring method, and source dates in the ledger until the desk brief ships.
What it does not cover
- Multi-file refactors or IDE agent loops.
- Private codebase context — problems are public contest slices only.
Scores
13 rowsEvidence ledger
1313 rows
91.7%best score
13source-checked
1sources
2026-07-21source date
Where the rows land
0255075100
Normalized to this benchmark's axis (0–100). Open the table for raw units.
Newest receipts
- Gemini 3 Pro91.7% ·
- GPT-5.288.9% ·
- GPT-5.186.8% ·
- OpenAI o380.8% ·
- Gemini 3 Flash Preview79.7% ·
Scores use this benchmark's own unit and axis, not a universal quality score.
| # | Model | Release date | Score | Provenance | Trust |
|---|---|---|---|---|---|
| 1 | Gemini 3 ProGoogle | 2025-11-18 | 91.7% | 2026-07-21 | Source-checked |
| 2 | GPT-5.2OpenAI | 2025-12-11 | 88.9% | 2026-07-21 | Source-checked |
| 3 | GPT-5.1OpenAI | 2025-11-13 | 86.8% | 2026-07-21 | Source-checked |
| 4 | OpenAI o3OpenAI | 2025-04-16 | 80.8% | 2026-07-21 | Source-checked |
| 5 | Gemini 3 Flash PreviewGoogle | 2025-12-17 | 79.7% | 2026-07-21 | Source-checked |
| 6 | DeepSeek R1DeepSeek | 2025-01-20 | 77% | 2026-07-21 | Source-checked |
| 7 | Claude Opus 4.5Anthropic | 2025-11-24 | 73.8% | 2026-07-21 | Source-checked |
| 8 | OpenAI o1OpenAI | 2024-09-12 | 67.9% | 2026-07-21 | Source-checked |
| 9 | Mistral Large 3Mistral | 2025-12-02 | 46.5% | 2026-07-21 | Source-checked |
| 10 | Llama 4 MaverickMeta | 2026-04-05 | 39.7% | 2026-07-21 | Source-checked |
| 11 | Llama 4 ScoutMeta | 2026-04-05 | 29.9% | 2026-07-21 | Source-checked |
| 12 | Command ACohere | 2026-03-15 | 28.7% | 2026-07-21 | Source-checked |
| 13 | Claude 3 OpusAnthropic | 2024-03-04 | 27.9% | 2026-07-21 | Source-checked |
What it covers
Data-quality note pending. Every ledger row still carries source URL, source date, and ingest timestamp.
Where it breaks down
No ceiling note is recorded yet. Treat clustering near the top as a warning that the benchmark may no longer separate frontier models.
Receipts
Sources and further reading
- AA LiveCodeBench· checked 2026-06-10
The Pack · Editorial newsletter
New cards in your inbox. Free.
One short email when a card ships or changes status. No tracking, no third-party analytics. Unsubscribe in one click.