Engineering: score by model
15 questions · the bar shows each score's 95% interval
| # | Model | Correct | Score | 95% interval |
|---|---|---|---|---|
| 1 |  Claude Fable 5.1Anthropic | 13 / 15 | 86.7% | 62–96% |
| 1 |  Gemini 4 ArgonGoogle · statistically tied with #1 | 13 / 15 | 86.7% | 62–96% |
| 1 | Claude Sonnet 5.5Anthropic · statistically tied with #1 | 13 / 15 | 86.7% | 62–96% |
| 4 | Claude Opus 5.5Anthropic · statistically tied with #1 | 12 / 15 | 80.0% | 55–93% |
| 4 | Gemini 3.8 FlashGoogle · statistically tied with #1 | 12 / 15 | 80.0% | 55–93% |
| 6 |  Muse WonderMeta · statistically tied with #1 | 11 / 15 | 73.3% | 48–89% |
| 6 |  Grok 4.7SpaceXAI · statistically tied with #1 | 11 / 15 | 73.3% | 48–89% |
| 6 |  GPT-5.6 TerraOpenAI · statistically tied with #1 | 11 / 15 | 73.3% | 48–89% |
| 6 | GPT-5.3-CodexOpenAI · statistically tied with #1 | 11 / 15 | 73.3% | 48–89% |
| 6 | Muse Spark OpenMeta · statistically tied with #1 | 11 / 15 | 73.3% | 48–89% |
| 6 |  Kimi K3.1 ProMoonshotAI · statistically tied with #1 | 11 / 15 | 73.3% | 48–89% |
| 12 | GPT-6 AstraOpenAI · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 | GPT-6.1 Terra ProOpenAI · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 |  DeepSeek V4 Flash Vision ExpDeepSeek · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 | Gemini 3.5 Flash LiteGoogle · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 | Gemini 3.1 Pro PreviewGoogle · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 |  GLM 5.3Z.ai · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 | GPT-6.1 SolOpenAI · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 | GPT-5.4OpenAI · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 12 | Claude Haiku 5.5Anthropic · statistically tied with #1 | 10 / 15 | 66.7% | 42–85% |
| 21 | GPT-6.1 TerraOpenAI · statistically tied with #1 | 9 / 15 | 60.0% | 36–80% |
| 21 | DeepSeek V4.1 Flash 1017DeepSeek · statistically tied with #1 | 9 / 15 | 60.0% | 36–80% |
| 21 | GPT-6 LunaOpenAI · statistically tied with #1 | 9 / 15 | 60.0% | 36–80% |
| 21 |  Qwen3.8 Max (0902)Qwen · statistically tied with #1 | 9 / 15 | 60.0% | 36–80% |
| 25 | o3OpenAI · statistically tied with #1 | 8 / 15 | 53.3% | 30–75% |
| 25 | GLM 5.3 FlashZ.ai · statistically tied with #1 | 8 / 15 | 53.3% | 30–75% |
| 25 | DeepSeek V4 Pro 0813DeepSeek · statistically tied with #1 | 8 / 15 | 53.3% | 30–75% |
| 25 | GPT-6 SolOpenAI · statistically tied with #1 | 8 / 15 | 53.3% | 30–75% |
| 25 |  MiMo-V2.6-ProXiaomi · statistically tied with #1 | 8 / 15 | 53.3% | 30–75% |
| 30 | Kimi K2.7 CodeMoonshotAI · statistically tied with #1 | 7 / 15 | 46.7% | 25–70% |
| 30 | MiMo-V2.6-FlashXiaomi · statistically tied with #1 | 7 / 15 | 46.7% | 25–70% |
| 32 | Llama 4 MaverickMeta · statistically tied with #1 | 6 / 14 | 42.9% | 21–67% |
| 33 |  InklingThinking Machines · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 |  Step 5 PreviewStepFun · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 |  Mercury 2.5Inception · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 |  Phi 4Microsoft · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 |  Hy3Tencent · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 | Qwen3 Coder NextQwen · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 | Inkling SmallThinking Machines · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 | Qwen3.8 27BQwen · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 |  Nemotron 3 UltraNVIDIA · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 |  MiniMax M3.1 FlashMiniMax · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 33 | gpt-oss-120bOpenAI · statistically tied with #1 | 6 / 15 | 40.0% | 20–64% |
| 44 | Qwen3.8 2.4T A95BQwen | 5 / 15 | 33.3% | 15–58% |
| 44 | MiniMax M3MiniMax | 5 / 15 | 33.3% | 15–58% |
| 46 | Qwen3.7 PlusQwen | 4 / 15 | 26.7% | 11–52% |
| 46 | Nemotron 3.5 Super VLNVIDIA | 4 / 15 | 26.7% | 11–52% |
| 46 | gpt-oss-20bOpenAI | 4 / 15 | 26.7% | 11–52% |
| 46 |  Ministral 3 8B 2512Mistral | 4 / 15 | 26.7% | 11–52% |
| 46 | Nemotron 3.5 LightningNVIDIA | 4 / 15 | 26.7% | 11–52% |
| 46 |  Granite 4.2 8BIBM | 4 / 15 | 26.7% | 11–52% |
| 52 |  Command ACohere | 3 / 15 | 20.0% | 7–45% |
| 52 | Mistral Large 4Mistral | 3 / 15 | 20.0% | 7–45% |
| 54 | Llama 4 ScoutMeta | 2 / 15 | 13.3% | 4–38% |
| 54 | Ministral 3 3B 2512Mistral | 2 / 15 | 13.3% | 4–38% |
| 54 |  Solar Mini 4Upstage | 2 / 15 | 13.3% | 4–38% |
| 54 | Ministral 3 14B 2512Mistral | 2 / 15 | 13.3% | 4–38% |
| 54 | Llama 3.3 70B InstructMeta | 2 / 15 | 13.3% | 4–38% |
| 54 | Qwen3.6 35B A3BQwen | 2 / 15 | 13.3% | 4–38% |
| 60 |  Ling 3.1 FlashinclusionAI | 1 / 15 | 6.7% | 1–30% |
| 60 | Granite 4.0 MicroIBM | 1 / 15 | 6.7% | 1–30% |
| 60 |  Nova Micro 1.0Amazon | 1 / 15 | 6.7% | 1–30% |
| 60 | Solar Pro 4Upstage | 1 / 15 | 6.7% | 1–30% |
| 60 |  LongCat 2.0Meituan | 1 / 15 | 6.7% | 1–30% |
| 65 | Mistral Medium 3.5Mistral | 0 / 15 | 0.0% | 0–20% |
| 65 |  Reka Flash 3rekaai | 0 / 15 | 0.0% | 0–20% |
| 65 | Gemma 4 26B A4BGoogle | 0 / 15 | 0.0% | 0–20% |
| 65 | Gemma 4 31BGoogle | 0 / 15 | 0.0% | 0–20% |
| 65 | Qwen3.5 397B A17BQwen | 0 / 15 | 0.0% | 0–20% |
| 65 |  Hermes 4 405BNous | 0 / 15 | 0.0% | 0–20% |
| 65 | Mistral Small 4Mistral | 0 / 15 | 0.0% | 0–20% |
| 65 |  Trinity Large ThinkingArcee AI | 0 / 15 | 0.0% | 0–20% |