MMLU-Pro: score by model
30 questions · the bar shows each score's 95% interval
| # | Model | Correct | Score | 95% interval |
|---|---|---|---|---|
| 1 |  Gemini 3.8 FlashGoogle | 29 / 30 | 96.7% | 83–99% |
| 1 | Gemini 4 ArgonGoogle · statistically tied with #1 | 29 / 30 | 96.7% | 83–99% |
| 3 |  Muse WonderMeta · statistically tied with #1 | 28 / 30 | 93.3% | 79–98% |
| 3 |  o3OpenAI · statistically tied with #1 | 28 / 30 | 93.3% | 79–98% |
| 3 |  Claude Opus 5.5Anthropic · statistically tied with #1 | 28 / 30 | 93.3% | 79–98% |
| 3 |  DeepSeek V4.1 Flash 1017DeepSeek · statistically tied with #1 | 28 / 30 | 93.3% | 79–98% |
| 3 |  Qwen3.8 Max (0902)Qwen · statistically tied with #1 | 28 / 30 | 93.3% | 79–98% |
| 8 |  Kimi K2.7 CodeMoonshotAI · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | Qwen3.7 PlusQwen · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | Claude Fable 5.1Anthropic · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 |  Grok 4.7SpaceXAI · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | GPT-5.6 TerraOpenAI · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | GPT-5.3-CodexOpenAI · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 |  GLM 5.3 FlashZ.ai · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | Qwen3 Coder NextQwen · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | Gemini 3.5 Flash LiteGoogle · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | DeepSeek V4 Pro 0813DeepSeek · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | Muse Spark OpenMeta · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | GLM 5.3Z.ai · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 |  MiMo-V2.6-ProXiaomi · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 8 | Claude Sonnet 5.5Anthropic · statistically tied with #1 | 27 / 30 | 90.0% | 74–97% |
| 22 |  InklingThinking Machines · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | GPT-6 AstraOpenAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | GPT-6.1 TerraOpenAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | GPT-6.1 Terra ProOpenAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | DeepSeek V4 Flash Vision ExpDeepSeek · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | Inkling SmallThinking Machines · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | Gemini 3.1 Pro PreviewGoogle · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | GPT-6.1 SolOpenAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | GPT-5.4OpenAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | GPT-6 SolOpenAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 22 | Kimi K3.1 ProMoonshotAI · statistically tied with #1 | 26 / 30 | 86.7% | 70–95% |
| 33 |  Step 5 PreviewStepFun · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 |  Mercury 2.5Inception · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 | Llama 3.3 70B InstructMeta · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 |  Hy3Tencent · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 |  Nemotron 3 UltraNVIDIA · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 | Claude Haiku 5.5Anthropic · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 | GPT-6 LunaOpenAI · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 33 |  MiniMax M3.1 FlashMiniMax · statistically tied with #1 | 25 / 30 | 83.3% | 66–93% |
| 41 |  Command ACohere · statistically tied with #1 | 24 / 30 | 80.0% | 63–90% |
| 41 |  Ministral 3 8B 2512Mistral · statistically tied with #1 | 24 / 30 | 80.0% | 63–90% |
| 43 | Llama 4 MaverickMeta · statistically tied with #1 | 23 / 29 | 79.3% | 62–90% |
| 44 | Qwen3.8 2.4T A95BQwen · statistically tied with #1 | 23 / 30 | 76.7% | 59–88% |
| 44 | gpt-oss-20bOpenAI · statistically tied with #1 | 23 / 30 | 76.7% | 59–88% |
| 44 | MiMo-V2.6-FlashXiaomi · statistically tied with #1 | 23 / 30 | 76.7% | 59–88% |
| 44 | MiniMax M3MiniMax · statistically tied with #1 | 23 / 30 | 76.7% | 59–88% |
| 44 | Qwen3.8 27BQwen · statistically tied with #1 | 23 / 30 | 76.7% | 59–88% |
| 44 | Mistral Large 4Mistral · statistically tied with #1 | 23 / 30 | 76.7% | 59–88% |
| 50 |  Solar Pro 4Upstage · statistically tied with #1 | 22 / 30 | 73.3% | 56–86% |
| 50 |  LongCat 2.0Meituan · statistically tied with #1 | 22 / 30 | 73.3% | 56–86% |
| 50 | gpt-oss-120bOpenAI · statistically tied with #1 | 22 / 30 | 73.3% | 56–86% |
| 53 |  Ling 3.1 FlashinclusionAI · statistically tied with #1 | 21 / 30 | 70.0% | 52–83% |
| 53 | Qwen3.5 397B A17BQwen · statistically tied with #1 | 21 / 30 | 70.0% | 52–83% |
| 53 |  Phi 4Microsoft · statistically tied with #1 | 21 / 30 | 70.0% | 52–83% |
| 53 | Qwen3.6 35B A3BQwen · statistically tied with #1 | 21 / 30 | 70.0% | 52–83% |
| 57 | Llama 4 ScoutMeta | 20 / 30 | 66.7% | 49–81% |
| 57 | Nemotron 3.5 Super VLNVIDIA | 20 / 30 | 66.7% | 49–81% |
| 57 | Solar Mini 4Upstage | 20 / 30 | 66.7% | 49–81% |
| 60 | Nemotron 3.5 LightningNVIDIA | 19 / 30 | 63.3% | 46–78% |
| 60 | Ministral 3 14B 2512Mistral | 19 / 30 | 63.3% | 46–78% |
| 60 | Mistral Small 4Mistral | 19 / 30 | 63.3% | 46–78% |
| 63 | Mistral Medium 3.5Mistral | 18 / 30 | 60.0% | 42–75% |
| 63 |  Hermes 4 405BNous | 18 / 30 | 60.0% | 42–75% |
| 63 |  Granite 4.2 8BIBM | 18 / 30 | 60.0% | 42–75% |
| 63 |  Trinity Large ThinkingArcee AI | 18 / 30 | 60.0% | 42–75% |
| 67 | Gemma 4 31BGoogle | 17 / 30 | 56.7% | 39–73% |
| 68 |  Nova Micro 1.0Amazon | 15 / 30 | 50.0% | 33–67% |
| 69 | Ministral 3 3B 2512Mistral | 14 / 30 | 46.7% | 30–64% |
| 70 | Granite 4.0 MicroIBM | 12 / 30 | 40.0% | 25–58% |
| 71 | Gemma 4 26B A4BGoogle | 11 / 30 | 36.7% | 22–54% |
| 72 |  Reka Flash 3rekaai | 0 / 30 | 0.0% | 0–11% |