VerdictPal
Join

Benchmarks / Reasoning

GPQA Diamond

Graduate-Level Google-Proof Q&A

GPQA Diamond explained: graduate-level Google-proof Q&A in biology, physics, and chemistry. Items are written so non-experts fail even with web search; Diamond is the expert-validated hard subset (198 multiple-choice questions). A high score means correct answers on that set under the published protocol — not a general research-assistant grade.

ReasoningSolidActiveLow contaminationSince 2023
What this does not measure
  • General usefulness — it is a narrow slice of frontier science, not everyday tasks.
  • Whether a model knows when it is out of its depth; it still answers every question.
  • Small size (198 Diamond questions) means a few lucky guesses move the score noticeably.
Analysis

Why this benchmark is useful

Use GPQA Diamond when you need a discriminating frontier-science check after easier suites saturate. Treat it as one reasoning receipt among others — useful for model comparison, not as a pick list for student writing tools.

Scope

Coverage map

Task family
Reasoning
Format
198 expert-validated multiple-choice questions in the Diamond set; designed to be 'Google-proof'.
Scoring
Accuracy (% correct).
Maintainer
Rein et al.
Reading guide

How to read the scores

Read accuracy on the 198 Diamond MCQs against published ceilings: PhD experts ~65–74% in-field, skilled non-experts with the web ~34%. A few lucky items move the percentage. Prefer dated, source-checked rows with a clear prompting setup; do not treat the leaderboard as everyday usefulness.

Blind spots

What it does not cover

  • General usefulness — it is a narrow slice of frontier science, not everyday tasks.
  • Whether a model knows when it is out of its depth; it still answers every question.
  • Small size (198 Diamond questions) means a few lucky guesses move the score noticeably.
Scores

Evidence ledger

184 rows
184184 rows
96.1%best score
177source-checked
2sources
2026-09-28to 2026-05-15
172

api · api

Papers / model cards12

manual-snapshot · manual

Distribution

Where the rows land

0255075100

Normalized to this benchmark's axis (0–100). Open the table for raw units.

Timeline

Newest receipts

  1. GPT-6 Astra96.1% ·
  2. Gemini 3.8 Flash95.3% ·
  3. Grok 4.694.9% ·
  4. Gemini 3.7 Flash94.5% ·
  5. Gemini 3.1 Pro Preview94.1% ·

Scores use this benchmark's own unit and axis, not a universal quality score.

#ModelRelease dateScoreProvenanceTrust
1GPT-6 AstraOpenAI2026-09-0396.1%
2026-09-28
Source-checked
2Gemini 3.8 FlashGoogle2026-09-0295.3%
2026-09-28
Source-checked
3Grok 4.6xAI2026-08-1294.9%
2026-09-28
Source-checked
1Claude Mythos PreviewAnthropic2026-06-0194.6%
GPQA Diamond paper leaderboard2026-06-02
Source-checked
5Gemini 3.7 FlashGoogle2026-08-1394.5%
2026-09-28
Source-checked
6Gemini 3.1 Pro PreviewGoogle2026-02-1994.1%
2026-09-28
Source-checked
7GPT-5.6 SolOpenAI2026-07-0994.1%
2026-09-28
Source-checked
8Claude Fable 5.1Anthropic2026-09-0193.7%
2026-09-28
Source-checked
9Claude Opus 5 (Adaptive Reasoning, High Effort)Anthropic2026-07-2493.7%
2026-09-28
Source-checked
10Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)Anthropic2026-07-2493.7%
2026-09-28
Source-checked
11Kimi K3Moonshot2026-07-1693.5%
2026-09-28
Source-checked
12Muse Spark 1.3Meta2026-09-0293.5%
2026-09-28
Source-checked
13Qwen3.8 2.4T A95BAlibaba2026-08-1293.5%
2026-09-28
Source-checked
14Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic2026-07-2493.2%
2026-09-28
Source-checked
15GPT-5.5 (high)OpenAI2026-04-2393.2%
2026-09-28
Source-checked
16Grok 4.5xAI2026-07-0893.1%
2026-09-28
Source-checked
17MiniMax M3MiniMax2026-05-3192.9%
2026-09-28
Source-checked
18DeepSeek V4 Pro 0813 (Reasoning, Max Effort)DeepSeek2026-08-1392.8%
2026-09-28
Source-checked
19Gemini 3.6 Flash (high)Google2026-07-2192.8%
2026-09-28
Source-checked
20Qwen3.8 MaxAlibaba2026-08-0392.8%
2026-09-28
Source-checked
21Claude Fable 5Anthropic2026-06-0992.6%
2026-09-28
Source-checked
22GPT-5.5 (medium)OpenAI2026-04-2392.6%
2026-09-28
Source-checked
23GPT-5.6 TerraOpenAI2026-07-0992.5%
2026-09-28
Source-checked
24Qwen3.7 MaxAlibaba2026-05-1992.3%
2026-09-28
Source-checked
25Qwen3.8-Flash-NextAlibaba2026-08-2692.3%
2026-09-28
Source-checked
26Gemini 3.5 Flash (medium)Google2026-05-1992.1%
2026-09-28
Source-checked
27GPT 5.4OpenAI2026-03-0592%
2026-09-28
Source-checked
28Claude Opus 5 (Adaptive Reasoning, Medium Effort)Anthropic2026-07-2491.9%
2026-09-28
Source-checked
29GLM-5.3Zhipu2026-08-1491.7%
2026-09-28
Source-checked
30GPT-5.3 Codex (xhigh)OpenAI2026-02-0591.5%
2026-09-28
Source-checked
31Claude Opus 4.7Anthropic2026-04-1691.4%
2026-09-28
Source-checked
32DeepSeek V4 Flash Vision (Reasoning, Max Effort)DeepSeek2026-08-2191.3%
2026-09-28
Source-checked
33GLM-5.3-FlashZhipu2026-08-2691.2%
2026-09-28
Source-checked
34Claude Sonnet 5Anthropic2026-06-3091.1%
2026-09-28
Source-checked
35GPT-5.6 LunaOpenAI2026-07-0991.1%
2026-09-28
Source-checked
36Grok 4.20 ReasoningxAI2026-03-0591.1%
2026-09-28
Source-checked
37GPT-5.5 (low)OpenAI2026-04-2391%
2026-09-28
Source-checked
38DeepSeek V4 Flash 0731 (Reasoning, Max Effort)DeepSeek2026-07-3190.8%
2026-09-28
Source-checked
39Gemini 3 ProGoogle2025-11-1890.8%
2026-09-28
Source-checked
1Kimi K2.6Moonshot2026-04-2090.5%
GPQA Diamond paper leaderboard2026-05-15
Source-checked
41Qwen3.8 27BAlibaba2026-08-1490.5%
2026-09-28
Source-checked
42Muse Spark 1.2Meta2026-08-0590.4%
2026-09-28
Source-checked
43GPT-5.2OpenAI2025-12-1190.3%
2026-09-28
Source-checked
44Qwen 3.7 PlusAlibaba2026-06-0290%
2026-09-28
Source-checked
45GPT-5.2 Codex (xhigh)OpenAI2025-12-1189.9%
2026-09-28
Source-checked
46Gemini 3 Flash Preview (Reasoning)Google2025-12-1789.8%
2026-09-28
Source-checked
47Muse Spark 1.1Meta2026-07-0989.8%
2026-09-28
Source-checked
48Hy3Other2026-07-0689.7%
2026-09-28
Source-checked
49Claude Opus 4.6 (Adaptive Reasoning, Max Effort)Anthropic2026-02-0589.6%
2026-09-28
Source-checked
50Kimi K2.7 CodeMoonshot2026-06-1289.6%
2026-09-28
Source-checked
51GLM-5.2Zhipu2026-06-1389.5%
2026-09-28
Source-checked
52Grok Build 0.1 0616xAI2026-06-1689.5%
2026-09-28
Source-checked
53Inkling SmallOther2026-07-3089.5%
2026-09-28
Source-checked
54Qwen3.5 397B A17B (Reasoning)Alibaba2026-02-1689.3%
2026-09-28
Source-checked
55Nex-N2-ProOther2026-06-0289.2%
2026-09-28
Source-checked
56Solar Pro 4Other2026-08-0689.1%
2026-09-28
Source-checked
57Grok 4.3 (medium)xAI2026-04-3089%
2026-09-28
Source-checked
58Claude Opus 5 (Adaptive Reasoning, Low Effort)Anthropic2026-07-2488.9%
2026-09-28
Source-checked
59Qwen3.6 Max PreviewAlibaba2026-04-2088.8%
2026-09-28
Source-checked
60Gemini 3 Pro Preview (low)Google2025-11-1888.7%
2026-09-28
Source-checked
61Claude Opus 4.7 (Non-reasoning, High Effort)Anthropic2026-04-1688.5%
2026-09-28
Source-checked
62Grok 4.20 0309 (Reasoning)xAI2026-03-1088.5%
2026-09-28
Source-checked
63Muse SparkOther2026-01-0188.4%
2026-09-28
Source-checked
64Qwen3.6 PlusAlibaba2026-04-0288.2%
2026-09-28
Source-checked
65GPT-5.4 MiniOpenAI2026-03-1787.5%
2026-09-28
Source-checked
66GPT-5.1OpenAI2025-11-1387.3%
2026-09-28
Source-checked
67GPT-5.4 (low)OpenAI2026-03-0587.1%
2026-09-28
Source-checked
68GLM-5.1Zhipu2026-04-0786.8%
2026-09-28
Source-checked
69GPT-5.2 (medium)OpenAI2025-12-1186.4%
2026-09-28
Source-checked
70GPT-5.1 Codex (high)OpenAI2025-11-1386%
2026-09-28
Source-checked
71GLM-4.7 (Reasoning)Zhipu2025-12-2285.9%
2026-09-28
Source-checked
72Gemma 4 31B ITGoogle2026-03-0185.7%
2026-09-28
Source-checked
73GPT-5 (high)OpenAI2025-08-0785.4%
2026-09-28
Source-checked
74GLM-5-TurboZhipu2026-03-1584.7%
2026-09-28
Source-checked
75GPT-5.5 Instant (May 2026)OpenAI2026-05-0584.6%
2026-09-28
Source-checked
76GPT-5 (medium)OpenAI2025-08-0784.2%
2026-09-28
Source-checked
77Gemini 3.5 Flash-LiteGoogle2026-07-2183.8%
2026-09-28
Source-checked
78GPT-5 Codex (high)OpenAI2025-09-2383.7%
2026-09-28
Source-checked
79Gemini 3.5 Flash (minimal)Google2026-05-1982.8%
2026-09-28
Source-checked
80GPT-5.5 Instant (June 2026)OpenAI2026-06-2582.3%
2026-09-28
Source-checked
81Gemini 3.1 Flash LiteGoogle2026-05-0782.2%
2026-09-28
Source-checked
82GLM-5 (Reasoning)Zhipu2026-02-1182%
2026-09-28
Source-checked
83GPT-5.4 NanoOpenAI2026-03-1781.7%
2026-09-28
Source-checked
84GLM 5V Turbo (Reasoning)Zhipu2026-04-0180.9%
2026-09-28
Source-checked
85GPT-5 (low)OpenAI2025-08-0780.8%
2026-09-28
Source-checked
1Gemini 3.1 ProGoogleCoT prompting. PhD-level science MCQ.2026-02-1978.4%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
2Grok 4.3xAI2026-04-3088%
GPQA Diamond paper leaderboard2026-05-15
Source-checked
88Kimi K2.5Moonshot2026-01-2787.9%
2026-09-28
Source-checked
89Grok 4xAI2025-07-1087.7%
2026-09-28
Source-checked
90Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)Anthropic2026-02-1787.5%
2026-09-28
Source-checked
91MiniMax M2.7MiniMax2026-04-1587.4%
2026-09-28
Source-checked
92K2 Horizon 375B A23BOther2026-09-0387.3%
2026-09-28
Source-checked
93Inkling (xhigh)Other2026-07-1587.2%
2026-09-28
Source-checked
94MiMo-V2-ProXiaomi2026-03-1887%
2026-09-28
Source-checked
95Motif 3 (Beta)Other2026-07-2186.9%
2026-09-28
Source-checked
96Hy3-preview (Reasoning)Other2026-04-2386.7%
2026-09-28
Source-checked
97Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIA2026-06-0486.7%
2026-09-28
Source-checked
98Claude Opus 4.5 (Reasoning)Anthropic2025-11-2486.6%
2026-09-28
Source-checked
99MiMo-V2.5-ProXiaomi2026-04-2286.6%
2026-09-28
Source-checked
100Apodex 1.1Other2026-08-3086.4%
2026-09-28
Source-checked
101Qwen3 Max ThinkingAlibaba2026-01-2686.1%
2026-09-28
Source-checked
102Qwen3.5 397B A17B (Non-reasoning)Alibaba2026-02-1686.1%
2026-09-28
Source-checked
103Qwen3.5 27B (Reasoning)Alibaba2026-02-2485.8%
2026-09-28
Source-checked
104A.X-K2Other2026-08-1285.7%
2026-09-28
Source-checked
105Qwen3.5 122B A10B (Reasoning)Alibaba2026-02-2485.7%
2026-09-28
Source-checked
106Ring-2.6-1TOther2026-05-0885.7%
2026-09-28
Source-checked
107Solar Open2 250BOther2026-08-1285.7%
2026-09-28
Source-checked
108KAT Coder Pro V2Other2026-03-2785.5%
2026-09-28
Source-checked
109Ling 3.0 FlashOther2026-08-0485.5%
2026-09-28
Source-checked
110MiMo-V2-Omni-0327Xiaomi2026-03-2785.5%
2026-09-28
Source-checked
111MiMo-V2.5Xiaomi2026-04-2284.9%
2026-09-28
Source-checked
112MiniMax M2.5MiniMax2026-04-0184.8%
2026-09-28
Source-checked
113MiMo-V2-Flash (Reasoning)Xiaomi2025-12-1684.6%
2026-09-28
Source-checked
114JT-4.1 Flash 236B A21BOther2026-07-0984.5%
2026-09-28
Source-checked
115o3-proOpenAI2025-06-1084.5%
2026-09-28
Source-checked
116Grok 4.3 (low)xAI2026-04-3084.3%
2026-09-28
Source-checked
117Kimi K3 (low)Moonshot2026-07-1684.2%
2026-09-28
Source-checked
118Qwen3.6 27B (Reasoning)Alibaba2026-04-2284.2%
2026-09-28
Source-checked
119Qwen3.6 35B A3B (Reasoning)Alibaba2026-04-1684.1%
2026-09-28
Source-checked
120Claude Opus 4.6Anthropic2026-02-0584%
2026-09-28
Source-checked
121DeepSeek V3.2 (Reasoning)DeepSeek2025-12-0184%
2026-09-28
Source-checked
122Kimi K2 ThinkingMoonshot2025-11-0683.8%
2026-09-28
Source-checked
123MiMo-V2-Flash (Feb 2026)Xiaomi2025-12-1683.5%
2026-09-28
Source-checked
124Muse GlimmerMeta2026-08-1083.5%
2026-09-28
Source-checked
125Claude 4.5 Sonnet (Reasoning)Anthropic2025-09-2983.4%
2026-09-28
Source-checked
126Motif 3Other2026-08-1283.4%
2026-09-28
Source-checked
127MiniMax-M2.1MiniMax2025-12-2383%
2026-09-28
Source-checked
128JT-35B-FlashOther2026-05-1482.9%
2026-09-28
Source-checked
129K-EXAONE 2.0 0803Other2026-08-1282.9%
2026-09-28
Source-checked
130Qwen3.6 27B (Non-reasoning)Alibaba2026-04-2282.9%
2026-09-28
Source-checked
131MiMo-V2-OmniXiaomi2026-03-1982.8%
2026-09-28
Source-checked
132OpenAI o3OpenAI2025-04-1682.7%
2026-09-28
Source-checked
133Qwen3.6 35B A3B (Non-reasoning)Alibaba2026-04-1681.7%
2026-09-28
Source-checked
134DeepSeek R1DeepSeek2025-01-2081.3%
2026-09-28
Source-checked
135Gemini 3 Flash PreviewGoogle2025-12-1781.2%
2026-09-28
Source-checked
136Claude Opus 4.5Anthropic2025-11-2481%
2026-09-28
Source-checked
137Claude 4.1 Opus (Reasoning)Anthropic2025-08-0580.9%
2026-09-28
Source-checked
138G9v3-39A5BOther2026-08-0380.5%
2026-09-28
Source-checked
139Claude Sonnet 4.6 (Non-reasoning, Low Effort)Anthropic2026-02-1779.7%
2026-09-28
Source-checked
140EXAONE 4.5 33BOther2026-04-0979.4%
2026-09-28
Source-checked
2Claude Sonnet 4.6Anthropic2026-02-1777.2%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
3GPT 5.5OpenAI2026-04-2381.2%
GPQA Diamond paper leaderboard2026-05-15
Source-checked
143Step 3.7 FlashStepFun2026-06-0180.9%
2026-09-28
Source-checked
3DeepSeek V4 Pro MaxDeepSeek2026-04-2475.6%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
4Claude Opus 4.8Anthropic2026-05-2879.8%
GPQA Diamond paper leaderboard2026-05-15
Source-checked
146Claude 4 Opus (Reasoning)Anthropic2025-05-2279.6%
2026-09-28
Source-checked
147Kimi K2.6 (Non-reasoning)Other2026-04-2078.8%
2026-09-28
Source-checked
148LongCat-2.0Meituan2026-06-2978%
2026-09-28
Source-checked
149Grok 4.20 0309 v2 (Non-reasoning)xAI2026-04-0777.6%
2026-09-28
Source-checked
150GPT-5.5 (Non-reasoning)OpenAI2026-04-2376.8%
2026-09-28
Source-checked
151MiMo-V2.5-Pro (Non-reasoning)Xiaomi2026-04-2276.2%
2026-09-28
Source-checked
152North Mini CodeCohere2026-06-0975.7%
2026-09-28
Source-checked
153Gemma 4 12B (Reasoning)Google2026-06-0775.3%
2026-09-28
Source-checked
154Ling-2.6-1TOther2026-04-2375.2%
2026-09-28
Source-checked
155Mistral Medium 3.5Mistral2026-04-2974.8%
2026-09-28
Source-checked
156OpenAI o1OpenAI2024-09-1274.7%
2026-09-28
Source-checked
157Nemotron 3.5 LightningNVIDIA2026-08-1174.3%
2026-09-28
Source-checked
4Gemini 3.5 FlashGoogle2026-05-1974.1%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
5Qwen 3.7 MaxAlibaba2026-05-2073.8%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
160Ling 3.0 TinyOther2026-08-0673.4%
2026-09-28
Source-checked
161Hy3-preview (Non-reasoning)Other2026-04-2373.2%
2026-09-28
Source-checked
6Mistral Large 3Mistral2025-12-0271.2%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
163MiniCPM5-2BOpenBMB2026-09-0770.2%
2026-09-28
Source-checked
7Llama 4 MaverickMeta2026-04-0569.4%
Artificial Analysis GPQA Diamond evaluation2026-06-09
Needs audit
165GLM-5.2 (Non-reasoning)Zhipu2026-06-1668.6%
2026-09-28
Source-checked
166JT-MINIOther2026-04-1567.6%
2026-09-28
Source-checked
167DiffusionGemma 26B A4BGoogle2026-06-1066.9%
2026-09-28
Source-checked
168GLM-5 (Non-reasoning)Zhipu2026-02-1166.6%
2026-09-28
Source-checked
169Gemma 4 12B (Non-reasoning)Google2026-06-1066.1%
2026-09-28
Source-checked
170Grok 4.3 (Non-reasoning)xAI2026-04-3065.8%
2026-09-28
Source-checked
171Granite 4.2 30BOther2026-08-2564.4%
2026-09-28
Source-checked
172Granite 4.2 8BOther2026-08-2563.1%
2026-09-28
Source-checked
173Ling 2.6 FlashOther2026-04-2159.3%
2026-09-28
Source-checked
174Llama 4 ScoutMeta2026-04-0558.7%
2026-09-28
Source-checked
175Granite 4.2 3BOther2026-08-2555.9%
2026-09-28
Source-checked
176Command ACohere2026-03-1552.7%
2026-09-28
Source-checked
177LFM2.5-8B-A1BLiquid2026-06-0851.3%
2026-09-28
Source-checked
178Claude 3 OpusAnthropic2024-03-0448.9%
2026-09-28
Source-checked
179Granite 4.1 30BOther2026-04-2948.1%
2026-09-28
Source-checked
180Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIA2026-04-2946.9%
2026-09-28
Source-checked
181G9v3-3BOther2026-07-2343.8%
2026-09-28
Source-checked
182MiniCPM-V 4.6 1.3BOpenBMB2026-05-1130.5%
2026-09-28
Source-checked
183MiniCPM5-1B (Reasoning)OpenBMB2026-06-0427.8%
2026-09-28
Source-checked
184MiniCPM5-1B (Non-reasoning)OpenBMB2026-05-2526.9%
2026-09-28
Source-checked
Method

What it covers

This benchmark sits in the reasoning family. It uses 198 expert-validated multiple-choice questions in the Diamond set; designed to be 'Google-proof'. The score should travel with its task format, scoring method, source date, and benchmark version.

Score ceiling

Where it breaks down

PhD-level domain experts score ~65-74% in their own field; skilled non-experts with web access ~34%.

Tools that report it

Receipts

Sources and further reading

← All benchmarksHow we score

Questions about this benchmark

Every answer below is assembled from the dated fields on this page. Nothing is written separately for search.

What does GPQA Diamond measure?

GPQA Diamond explained: graduate-level Google-proof Q&A in biology, physics, and chemistry. Items are written so non-experts fail even with web search; Diamond is the expert-validated hard subset (198 multiple-choice questions). A high score means correct answers on that set under the published protocol — not a general research-assistant grade.

What does a high GPQA Diamond score not prove?

A strong GPQA Diamond result says nothing about:

  • General usefulness — it is a narrow slice of frontier science, not everyday tasks.
  • Whether a model knows when it is out of its depth; it still answers every question.
  • Small size (198 Diamond questions) means a few lucky guesses move the score noticeably.

How is GPQA Diamond scored?

Accuracy (% correct). Task format: 198 expert-validated multiple-choice questions in the Diamond set; designed to be 'Google-proof'.

Is GPQA Diamond saturated?

GPQA Diamond is currently marked Active in the atlas. Ceiling context: PhD-level domain experts score ~65-74% in their own field; skilled non-experts with web access ~34%.

Can GPQA Diamond results be contaminated by training data?

Contamination risk for GPQA Diamond is graded Low contamination. Treat every row on this page as a public claim with a source and a date, not as a controlled experiment.

The Pack · Editorial newsletter

Know when the rules move.

One short email when a university changes its AI rule for theses, or a research tool changes its pricing or privacy terms. No tracking, no third-party analytics. Unsubscribe in one click.

The thesis AI checklist is free to use, no signup needed →