VerdictPal
Join

Benchmarks / Agentic

Terminal-Bench

Terminal-Bench 2.1

Whether an agent can complete realistic, containerized terminal tasks (Terminal-Bench 2.1 on the public AA API): debugging code, handling files, running commands, and producing the artifact the hidden checker expects. Index v4.2 weighted this suite at 10%; Index v4.3 uses Terminal-Bench v4.0 in the composite instead.

AgenticPrimary · AgenticSolidActiveLow contaminationSince 2025
What this does not measure
  • The model alone — harness, tools, retries, time limits, and container setup can move scores dramatically.
  • Code maintainability — a passed task can still produce brittle scripts or unreviewable patches.
  • Every developer workflow — terminal tasks are closer to agentic operations than product design, frontend polish, or long-running repository stewardship.
Analysis

Why this benchmark is useful

When you need realistic containerized terminal work — debug, files, commands, artifacts — as an agentic ops signal. This page tracks AA's published 2.1 / Hard rows, not the v4.0 slice inside Index v4.3.

Scope

Coverage map

Task family
Agentic
Format
Containerized tasks with a terminal environment; agents operate through commands and are graded by task-specific checks.
Scoring
Resolution/pass rate across audited tasks. Always read the agent harness and model pairing.
Maintainer
Laude Institute / Terminal-Bench
Reading guide

How to read the scores

Resolution/pass rate on audited Terminal-Bench 2.1 tasks from the AA API (`terminalbench_v2_1` / Hard). Frontier 2.1 pass rates sit in the high 80s to low 90s — that is the easier suite. Index v4.3's coding slice uses Terminal-Bench v4.0 (AA reported Astra max 59.1% vs Fable 5.1 52.0% on 7 Sep 2026), but that key is not in the 10 Sep 2026 API payload, so this ledger stays on 2.1 until AA publishes v4 rows. Always pair the model with its harness, tools, retries, and time limits.

Blind spots

What it does not cover

  • The model alone — harness, tools, retries, time limits, and container setup can move scores dramatically.
  • Code maintainability — a passed task can still produce brittle scripts or unreviewable patches.
  • Every developer workflow — terminal tasks are closer to agentic operations than product design, frontend polish, or long-running repository stewardship.
Scores

Evidence ledger

167 rows
167167 rows
91.39%best score
166source-checked
2sources
2026-09-28to 2026-06-29
166

api · api

Vendor claims1

manual-snapshot · manual

Distribution

Where the rows land

0255075100

Normalized to this benchmark's axis (0–100). Open the table for raw units.

Timeline

Newest receipts

  1. Claude Fable 5.191.39% ·
  2. Claude Opus 5 (Adaptive Reasoning, Max Effort)89.14% ·
  3. Qwen3.8 Max88.76% ·
  4. GPT-6 Astra88.39% ·
  5. Grok 4.688.39% ·

Scores use this benchmark's own unit and axis, not a universal quality score.

#ModelRelease dateScoreProvenanceTrust
1Claude Fable 5.1Anthropic2026-09-0191.39%
2026-09-28
Source-checked
2Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic2026-07-2489.14%
2026-09-28
Source-checked
3Qwen3.8 MaxAlibaba2026-08-0388.76%
2026-09-28
Source-checked
4GPT-6 AstraOpenAI2026-09-0388.39%
2026-09-28
Source-checked
5Grok 4.6xAI2026-08-1288.39%
2026-09-28
Source-checked
6Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)Anthropic2026-07-2488.01%
2026-09-28
Source-checked
7GPT-5.6 SolOpenAI2026-07-0988.01%
2026-09-28
Source-checked
8GPT-5.6 TerraOpenAI2026-07-0988.01%
2026-09-28
Source-checked
9Claude Opus 5 (Adaptive Reasoning, High Effort)Anthropic2026-07-2487.64%
2026-09-28
Source-checked
10Gemini 3.8 FlashGoogle2026-09-0287.64%
2026-09-28
Source-checked
11Claude Opus 5 (Adaptive Reasoning, Medium Effort)Anthropic2026-07-2486.14%
2026-09-28
Source-checked
12Qwen3.8-Flash-NextAlibaba2026-08-2686.14%
2026-09-28
Source-checked
13Gemini 3.7 FlashGoogle2026-08-1385.77%
2026-09-28
Source-checked
14Kimi K3Moonshot2026-07-1685.02%
2026-09-28
Source-checked
15Claude Fable 5Anthropic2026-06-0984.64%
2026-09-28
Source-checked
16GLM-5.3-FlashZhipu2026-08-2684.27%
2026-09-28
Source-checked
17Muse Spark 1.3Meta2026-09-0284.27%
2026-09-28
Source-checked
18GLM-5.3Zhipu2026-08-1483.9%
2026-09-28
Source-checked
19Kimi K3 (low)Moonshot2026-07-1682.4%
2026-09-28
Source-checked
20Qwen3.8 2.4T A95BAlibaba2026-08-1282.02%
2026-09-28
Source-checked
21Grok 4.5xAI2026-07-0881.65%
2026-09-28
Source-checked
22GPT-5.6 LunaOpenAI2026-07-0980.9%
2026-09-28
Source-checked
23Claude Sonnet 5Anthropic2026-06-3080.52%
2026-09-28
Source-checked
24GPT-5.5 (medium)OpenAI2026-04-2380.52%
2026-09-28
Source-checked
25Muse Spark 1.2Meta2026-08-0580.15%
2026-09-28
Source-checked
26Qwen3.8 27BAlibaba2026-08-1479.78%
2026-09-28
Source-checked
27GPT-5.5 (high)OpenAI2026-04-2379.4%
2026-09-28
Source-checked
28DeepSeek V4 Flash 0731 (Reasoning, Max Effort)DeepSeek2026-07-3178.65%
2026-09-28
Source-checked
29DeepSeek V4 Pro 0813 (Reasoning, Max Effort)DeepSeek2026-08-1378.65%
2026-09-28
Source-checked
30GLM-5.2Zhipu2026-06-1377.9%
2026-09-28
Source-checked
31Muse Spark 1.1Meta2026-07-0977.9%
2026-09-28
Source-checked
32Gemini 3.6 Flash (high)Google2026-07-2177.53%
2026-09-28
Source-checked
33Claude Opus 5 (Adaptive Reasoning, Low Effort)Anthropic2026-07-2476.4%
2026-09-28
Source-checked
34Motif 3Other2026-08-1274.91%
2026-09-28
Source-checked
35Qwen3.7 MaxAlibaba2026-05-1974.53%
2026-09-28
Source-checked
36DeepSeek V4 Flash Vision (Reasoning, Max Effort)DeepSeek2026-08-2174.16%
2026-09-28
Source-checked
37Gemini 3.1 Pro PreviewGoogle2026-02-1973.78%
2026-09-28
Source-checked
38K2 Horizon 375B A23BOther2026-09-0371.91%
2026-09-28
Source-checked
39Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)Anthropic2026-02-1771.16%
2026-09-28
Source-checked
40LongCat-2.0MeituanVendor-reported LongCat-2.0 score; keep separate from audited Terminal-Bench leaderboard rows until an independent run lands.2026-06-2970.8%
Meituan LongCat-2.0 release materials2026-06-29
Needs audit
41Motif 3 (Beta)Other2026-07-2170.79%
2026-09-28
Source-checked
42KAT Coder Pro V2Other2026-03-2770.04%
2026-09-28
Source-checked
43Apodex 1.1Other2026-08-3069.66%
2026-09-28
Source-checked
44Nex-N2-ProOther2026-06-0267.79%
2026-09-28
Source-checked
45Kimi K2.7 CodeMoonshot2026-06-1267.42%
2026-09-28
Source-checked
46GPT-5.5 (low)OpenAI2026-04-2365.54%
2026-09-28
Source-checked
47MiMo-V2.5-ProXiaomi2026-04-2265.17%
2026-09-28
Source-checked
48Hy3Other2026-07-0664.42%
2026-09-28
Source-checked
49MiMo-V2.5Xiaomi2026-04-2263.67%
2026-09-28
Source-checked
50Muse SparkOther2026-01-0162.17%
2026-09-28
Source-checked
51GLM-5.1Zhipu2026-04-0761.8%
2026-09-28
Source-checked
52Qwen3.6 PlusAlibaba2026-04-0261.42%
2026-09-28
Source-checked
53GPT-5.5 (Non-reasoning)OpenAI2026-04-2361.05%
2026-09-28
Source-checked
54Qwen 3.7 PlusAlibaba2026-06-0261.05%
2026-09-28
Source-checked
55GPT-5.4 NanoOpenAI2026-03-1760.67%
2026-09-28
Source-checked
56Qwen3.6 27B (Reasoning)Alibaba2026-04-2260.67%
2026-09-28
Source-checked
57JT-4.1 Flash 236B A21BOther2026-07-0959.55%
2026-09-28
Source-checked
58GPT-5.4 MiniOpenAI2026-03-1759.18%
2026-09-28
Source-checked
59Solar Pro 4Other2026-08-0657.3%
2026-09-28
Source-checked
60Claude 4.5 Sonnet (Reasoning)Anthropic2025-09-2955.81%
2026-09-28
Source-checked
61Ling 3.0 FlashOther2026-08-0455.43%
2026-09-28
Source-checked
62MiniMax M2.7MiniMax2026-04-1555.43%
2026-09-28
Source-checked
63Inkling (xhigh)Other2026-07-1555.06%
2026-09-28
Source-checked
64Inkling SmallOther2026-07-3055.06%
2026-09-28
Source-checked
65Claude Opus 4.7 (Non-reasoning, High Effort)Anthropic2026-04-1654.55%
2026-09-28
Source-checked
66Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIA2026-06-0453.93%
2026-09-28
Source-checked
67Gemini 3.5 Flash-LiteGoogle2026-07-2153.56%
2026-09-28
Source-checked
68GPT-5.3 Codex (xhigh)OpenAI2026-02-0553.03%
2026-09-28
Source-checked
69GPT-5.1OpenAI2025-11-1352.43%
2026-09-28
Source-checked
70Grok Build 0.1 0616xAI2026-06-1652.06%
2026-09-28
Source-checked
71GLM-5.2 (Non-reasoning)Zhipu2026-06-1651.69%
2026-09-28
Source-checked
72Muse GlimmerMeta2026-08-1051.69%
2026-09-28
Source-checked
73Qwen3.5 397B A17B (Reasoning)Alibaba2026-02-1651.31%
2026-09-28
Source-checked
74Qwen3.6 27B (Non-reasoning)Alibaba2026-04-2251.31%
2026-09-28
Source-checked
75Mistral Medium 3.5Mistral2026-04-2950.56%
2026-09-28
Source-checked
76Qwen3.5 122B A10B (Reasoning)Alibaba2026-02-2447.57%
2026-09-28
Source-checked
77Claude Opus 4.5 (Reasoning)Anthropic2025-11-2446.97%
2026-09-28
Source-checked
78GPT-5.2OpenAI2025-12-1146.97%
2026-09-28
Source-checked
79DeepSeek V3.2 (Reasoning)DeepSeek2025-12-0146.82%
2026-09-28
Source-checked
80Claude Opus 4.6 (Adaptive Reasoning, Max Effort)Anthropic2026-02-0546.21%
2026-09-28
Source-checked
81Gemini 3.5 Flash (minimal)Google2026-05-1946.21%
2026-09-28
Source-checked
82Kimi K2.5Moonshot2026-01-2745.69%
2026-09-28
Source-checked
83GLM-4.7 (Reasoning)Zhipu2025-12-2245.32%
2026-09-28
Source-checked
84Qwen3.6 35B A3B (Reasoning)Alibaba2026-04-1644.94%
2026-09-28
Source-checked
85Solar Open2 250BOther2026-08-1244.19%
2026-09-28
Source-checked
86Qwen3.6 Max PreviewAlibaba2026-04-2043.94%
2026-09-28
Source-checked
87Gemma 4 31B ITGoogle2026-03-0143.45%
2026-09-28
Source-checked
88GLM-5 (Reasoning)Zhipu2026-02-1143.18%
2026-09-28
Source-checked
89GPT-5.2 (medium)OpenAI2025-12-1143.18%
2026-09-28
Source-checked
90GPT-5.4 (low)OpenAI2026-03-0543.18%
2026-09-28
Source-checked
91Ring-2.6-1TOther2026-05-0843.07%
2026-09-28
Source-checked
92Claude Sonnet 4.6 (Non-reasoning, Low Effort)Anthropic2026-02-1742.42%
2026-09-28
Source-checked
93GPT-5.5 Instant (May 2026)OpenAI2026-05-0542.42%
2026-09-28
Source-checked
94Gemini 3 ProGoogle2025-11-1841.67%
2026-09-28
Source-checked
95Qwen3.6 35B A3B (Non-reasoning)Alibaba2026-04-1641.57%
2026-09-28
Source-checked
96Claude Opus 4.5Anthropic2025-11-2440.91%
2026-09-28
Source-checked
97Grok 4.20 0309 (Reasoning)xAI2026-03-1040.91%
2026-09-28
Source-checked
98MiMo-V2-ProXiaomi2026-03-1840.91%
2026-09-28
Source-checked
99K-EXAONE 2.0 0803Other2026-08-1240.45%
2026-09-28
Source-checked
100Gemini 3.5 Flash (medium)Google2026-05-1939.39%
2026-09-28
Source-checked
101GLM-5 (Non-reasoning)Zhipu2026-02-1139.39%
2026-09-28
Source-checked
102Step 3.7 FlashStepFun2026-06-0139.33%
2026-09-28
Source-checked
103A.X-K2Other2026-08-1238.95%
2026-09-28
Source-checked
104Gemini 3 Flash Preview (Reasoning)Google2025-12-1738.64%
2026-09-28
Source-checked
105GPT-5 (medium)OpenAI2025-08-0737.88%
2026-09-28
Source-checked
106GPT-5 Codex (high)OpenAI2025-09-2337.88%
2026-09-28
Source-checked
107Grok 4xAI2025-07-1037.88%
2026-09-28
Source-checked
108Grok 4.20 ReasoningxAI2026-03-0537.88%
2026-09-28
Source-checked
109Kimi K2.6 (Non-reasoning)Other2026-04-2037.88%
2026-09-28
Source-checked
110GPT-5.2 Codex (xhigh)OpenAI2025-12-1137.12%
2026-09-28
Source-checked
111OpenAI o3OpenAI2025-04-1637.12%
2026-09-28
Source-checked
112MiMo-V2-Omni-0327Xiaomi2026-03-2735.61%
2026-09-28
Source-checked
113MiMo-V2.5-Pro (Non-reasoning)Xiaomi2026-04-2235.61%
2026-09-28
Source-checked
114Qwen3.5 397B A17B (Non-reasoning)Alibaba2026-02-1635.61%
2026-09-28
Source-checked
115North Mini CodeCohere2026-06-0935.58%
2026-09-28
Source-checked
116GPT-5 (high)OpenAI2025-08-0735.21%
2026-09-28
Source-checked
117GPT-5.1 Codex (high)OpenAI2025-11-1334.85%
2026-09-28
Source-checked
118MiMo-V2-OmniXiaomi2026-03-1934.85%
2026-09-28
Source-checked
119MiniMax M2.5MiniMax2026-04-0134.85%
2026-09-28
Source-checked
120GPT-5.5 Instant (June 2026)OpenAI2026-06-2534.83%
2026-09-28
Source-checked
121Claude 4.1 Opus (Reasoning)Anthropic2025-08-0534.34%
2026-09-28
Source-checked
122Gemini 3 Pro Preview (low)Google2025-11-1834.09%
2026-09-28
Source-checked
123Hy3-preview (Reasoning)Other2026-04-2334.09%
2026-09-28
Source-checked
124Grok 4.3 (Non-reasoning)xAI2026-04-3034.08%
2026-09-28
Source-checked
125GLM-5-TurboZhipu2026-03-1533.33%
2026-09-28
Source-checked
126G9v3-39A5BOther2026-08-0332.58%
2026-09-28
Source-checked
127GLM 5V Turbo (Reasoning)Zhipu2026-04-0132.58%
2026-09-28
Source-checked
128Qwen3.5 27B (Reasoning)Alibaba2026-02-2432.58%
2026-09-28
Source-checked
129Gemini 3 Flash PreviewGoogle2025-12-1731.82%
2026-09-28
Source-checked
130Hy3-preview (Non-reasoning)Other2026-04-2331.82%
2026-09-28
Source-checked
131Gemini 3.1 Flash LiteGoogle2026-05-0731.09%
2026-09-28
Source-checked
132Claude 4 Opus (Reasoning)Anthropic2025-05-2231.06%
2026-09-28
Source-checked
133Kimi K2 ThinkingMoonshot2025-11-0631.06%
2026-09-28
Source-checked
134Ling-2.6-1TOther2026-04-2331.06%
2026-09-28
Source-checked
135MiMo-V2-Flash (Feb 2026)Xiaomi2025-12-1631.06%
2026-09-28
Source-checked
136Grok 4.3 (medium)xAI2026-04-3030.3%
2026-09-28
Source-checked
137JT-35B-FlashOther2026-05-1428.79%
2026-09-28
Source-checked
138MiniMax-M2.1MiniMax2025-12-2328.79%
2026-09-28
Source-checked
139MiMo-V2-Flash (Reasoning)Xiaomi2025-12-1628.03%
2026-09-28
Source-checked
140Ling 3.0 TinyOther2026-08-0627.72%
2026-09-28
Source-checked
141Gemma 4 12B (Reasoning)Google2026-06-0727.34%
2026-09-28
Source-checked
142Granite 4.2 30BOther2026-08-2526.59%
2026-09-28
Source-checked
143GPT-5 (low)OpenAI2025-08-0726.52%
2026-09-28
Source-checked
144Grok 4.3 (low)xAI2026-04-3026.52%
2026-09-28
Source-checked
145Ling 2.6 FlashOther2026-04-2124.34%
2026-09-28
Source-checked
146Nemotron 3.5 LightningNVIDIA2026-08-1124.34%
2026-09-28
Source-checked
147Qwen3 Max ThinkingAlibaba2026-01-2624.24%
2026-09-28
Source-checked
148EXAONE 4.5 33BOther2026-04-0921.35%
2026-09-28
Source-checked
149Granite 4.2 8BOther2026-08-2518.35%
2026-09-28
Source-checked
150JT-MINIOther2026-04-1518.18%
2026-09-28
Source-checked
151Grok 4.20 0309 v2 (Non-reasoning)xAI2026-04-0716.67%
2026-09-28
Source-checked
152DeepSeek R1DeepSeek2025-01-2015.91%
2026-09-28
Source-checked
153Granite 4.2 3BOther2026-08-2513.86%
2026-09-28
Source-checked
154OpenAI o1OpenAI2024-09-1212.88%
2026-09-28
Source-checked
155DiffusionGemma 26B A4BGoogle2026-06-1012.36%
2026-09-28
Source-checked
156Mistral Large 3Mistral2025-12-0211.99%
2026-09-28
Source-checked
157Gemma 4 12B (Non-reasoning)Google2026-06-1011.36%
2026-09-28
Source-checked
158MiniCPM5-2BOpenBMB2026-09-078.61%
2026-09-28
Source-checked
159Llama 4 MaverickMeta2026-04-057.87%
2026-09-28
Source-checked
160Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIA2026-04-296.74%
2026-09-28
Source-checked
161G9v3-3BOther2026-07-235.99%
2026-09-28
Source-checked
162LFM2.5-8B-A1BLiquid2026-06-084.55%
2026-09-28
Source-checked
163Granite 4.1 30BOther2026-04-292.62%
2026-09-28
Source-checked
164Command ACohere2026-03-150.76%
2026-09-28
Source-checked
165MiniCPM-V 4.6 1.3BOpenBMB2026-05-110%
2026-09-28
Source-checked
166MiniCPM5-1B (Non-reasoning)OpenBMB2026-05-250%
2026-09-28
Source-checked
167MiniCPM5-1B (Reasoning)OpenBMB2026-06-040%
2026-09-28
Source-checked
Method

What it covers

This benchmark sits in the agentic family. It uses Containerized tasks with a terminal environment; agents operate through commands and are graded by task-specific checks. The score should travel with its task format, scoring method, source date, and benchmark version.

Score ceiling

Where it breaks down

No ceiling note is recorded yet. Treat clustering near the top as a warning that the benchmark may no longer separate frontier models.

Tools that report it

Receipts

Sources and further reading

← All benchmarksHow we score

Questions about this benchmark

Every answer below is assembled from the dated fields on this page. Nothing is written separately for search.

What does Terminal-Bench measure?

Whether an agent can complete realistic, containerized terminal tasks (Terminal-Bench 2.1 on the public AA API): debugging code, handling files, running commands, and producing the artifact the hidden checker expects. Index v4.2 weighted this suite at 10%; Index v4.3 uses Terminal-Bench v4.0 in the composite instead.

What does a high Terminal-Bench score not prove?

A strong Terminal-Bench result says nothing about:

  • The model alone — harness, tools, retries, time limits, and container setup can move scores dramatically.
  • Code maintainability — a passed task can still produce brittle scripts or unreviewable patches.
  • Every developer workflow — terminal tasks are closer to agentic operations than product design, frontend polish, or long-running repository stewardship.

How is Terminal-Bench scored?

Resolution/pass rate across audited tasks. Always read the agent harness and model pairing. Task format: Containerized tasks with a terminal environment; agents operate through commands and are graded by task-specific checks.

Is Terminal-Bench saturated?

Terminal-Bench is currently marked Active in the atlas.

Can Terminal-Bench results be contaminated by training data?

Contamination risk for Terminal-Bench is graded Low contamination. Treat every row on this page as a public claim with a source and a date, not as a controlled experiment.

The Pack · Editorial newsletter

Know when the rules move.

One short email when a university changes its AI rule for theses, or a research tool changes its pricing or privacy terms. No tracking, no third-party analytics. Unsubscribe in one click.

The thesis AI checklist is free to use, no signup needed →