Why this benchmark is useful
It gives a crisp reasoning signal with right-or-wrong answers. Use it to compare structured problem solving, while watching sample size and pass@k settings.
Benchmarks / Math
American Invitational Mathematics Examination
AIME
Hard high-school competition math: 15 integer-answer problems per exam. Adopted as a frontier reasoning eval because each year's fresh paper resists contamination — until it leaks.
MathSolidDefunctMedium contamination riskSince 2024
What this does not measure
- Tiny sample (15 problems/year) — one problem is ~6.7%, so single-run scores are noisy; demand pass@k or multiple seeds.
- General capability — strong AIME scores correlate with reasoning training, not with everyday helpfulness.
- Which year — '90% on AIME' is meaningless without the exam year and number of attempts.
Coverage map
- Task family
- Math
- Format
- 15 problems, integer answers 0-999; reported as accuracy on AIME 2024/2025 papers.
- Scoring
- Accuracy (often pass@1 or cons@64).
- Maintainer
- MAA (exam) · used as an LLM eval
How to read the scores
Read AIME as a defunct signal with medium contamination risk. Compare models only when the source uses the same harness, prompting setup, sampling policy, and score unit.
What it does not cover
- Tiny sample (15 problems/year) — one problem is ~6.7%, so single-run scores are noisy; demand pass@k or multiple seeds.
- General capability — strong AIME scores correlate with reasoning training, not with everyday helpfulness.
- Which year — '90% on AIME' is meaningless without the exam year and number of attempts.
Scores
19 rowsEvidence ledger
1919 rows
95.67%best score
13source-checked
3sources
2026-07-21to 2024-09-12
Where the rows land
0255075100
Normalized to this benchmark's axis (0–100). Open the table for raw units.
Newest receipts
- Gemini 3 Pro95.67% ·
- GPT-5.194% ·
- OpenAI o388.33% ·
- Claude Opus 4.562.67% ·
- Gemini 3 Flash Preview55.67% ·
Scores use this benchmark's own unit and axis, not a universal quality score.
| # | Model | Release date | Score | Provenance | Trust |
|---|---|---|---|---|---|
| 1 | Gemini 3 ProGoogle | 2025-11-18 | 95.67% | 2026-07-21 | Source-checked |
| 1 | Claude Opus 4.8AnthropicAIME 2024 with extended thinking. | 2026-05-28 | 94.2% | Anthropic Claude 4 family2026-05-28 | Needs audit |
| 3 | GPT-5.1OpenAI | 2025-11-13 | 94% | 2026-07-21 | Source-checked |
| 2 | Claude 3.7 Sonnet (2025-02)AnthropicAIME 2024 with extended thinking enabled. | 2025-02-19 | 93.7% | Anthropic Claude 3.7 Sonnet announcement2025-02-19 | Source-checked |
| 3 | GPT 5.5OpenAI | 2026-04-23 | 91.8% | OpenAI model release notes2026-04-23 | Needs audit |
| 4 | GPT 5OpenAI | 2025-12-11 | 89.2% | OpenAI model release notes2025-12-01 | Needs audit |
| 5 | DeepSeek R1DeepSeekAIME 2024 with chain-of-thought. | 2025-01-20 | 88.4% | DeepSeek R1 model card2025-01-20 | Needs audit |
| 8 | OpenAI o3OpenAI | 2025-04-16 | 88.33% | 2026-07-21 | Source-checked |
| 6 | Gemini 3.1 ProGoogle | 2026-02-19 | 87.6% | Google Gemini 3.1 Pro2026-02-19 | Needs audit |
| 7 | o3 (cons@64, AIME 2025)OpenAIAIME 2025 paper; 64-sample consensus. Single-shot is markedly lower. | 2025-04-16 | 87.5% | OpenAI: Learning to Reason2025-01-20 | Source-checked |
| 9 | Kimi K2.6Moonshot | 2026-04-20 | 85.2% | Moonshot Kimi K2.62026-04-20 | Needs audit |
| 10 | OpenAI o1 (AIME 2024, cons@64)OpenAIWith 64-sample consensus; single-shot is markedly lower. | 2024-09-12 | 83.3% | OpenAI: Learning to Reason2024-09-12 | Source-checked |
| 13 | Claude Opus 4.5Anthropic | 2025-11-24 | 62.67% | 2026-07-21 | Source-checked |
| 14 | Gemini 3 Flash PreviewGoogle | 2025-12-17 | 55.67% | 2026-07-21 | Source-checked |
| 15 | Mistral Large 3Mistral | 2025-12-02 | 38% | 2026-07-21 | Source-checked |
| 16 | Llama 4 MaverickMeta | 2026-04-05 | 19.33% | 2026-07-21 | Source-checked |
| 17 | Llama 4 ScoutMeta | 2026-04-05 | 14% | 2026-07-21 | Source-checked |
| 18 | Command ACohere | 2026-03-15 | 13% | 2026-07-21 | Source-checked |
| 19 | Claude 3 OpusAnthropic | 2024-03-04 | 3.33% | 2026-07-21 | Source-checked |
What it covers
Archived — GPT-5.2 and peers now report 100% on recent AIME papers with extended thinking. Treat as solved, not a frontier separator.
Where it breaks down
No ceiling note is recorded yet. Treat clustering near the top as a warning that the benchmark may no longer separate frontier models.
Receipts
Sources and further reading
- OpenAI o1 announcement· checked 2026-05-30
The Pack · Editorial newsletter
New cards in your inbox. Free.
One short email when a card ships or changes status. No tracking, no third-party analytics. Unsubscribe in one click.