ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Subject · tested 27 Oct 2026

Best AI models for medicine

Scores on the 11 medicine questions in the suite (8 from SuperGPQA, 3 from MMLU-Pro). That is a small sample: the bars show each score's 95% interval, and overlapping bars mean the models can't be told apart.

Questions11of 80 in the suite
Top score7/11Claude Fable 5.1, o3, GPT-5.3-Codex, Gemini 3.5 Flash Lite, Gemini 3.1 Pro Preview, Claude Opus 5.5, Gemini 4 Argon, Claude Sonnet 5.5
Tied with the leader71overlapping 95% intervals
Median score45.5%of 72 models

Medicine: score by model

11 questions · the bar shows each score's 95% interval

Medicine leaderboard
#ModelCorrectScore95% interval
1Claude Fable 5.1Anthropic7 / 1163.6%35–85%
1o3OpenAI · statistically tied with #17 / 1163.6%35–85%
1GPT-5.3-CodexOpenAI · statistically tied with #17 / 1163.6%35–85%
1Gemini 3.5 Flash LiteGoogle · statistically tied with #17 / 1163.6%35–85%
1Gemini 3.1 Pro PreviewGoogle · statistically tied with #17 / 1163.6%35–85%
1Claude Opus 5.5Anthropic · statistically tied with #17 / 1163.6%35–85%
1Gemini 4 ArgonGoogle · statistically tied with #17 / 1163.6%35–85%
1Claude Sonnet 5.5Anthropic · statistically tied with #17 / 1163.6%35–85%
9InklingThinking Machines · statistically tied with #16 / 1154.5%28–79%
9Command ACohere · statistically tied with #16 / 1154.5%28–79%
9Llama 4 ScoutMeta · statistically tied with #16 / 1154.5%28–79%
9Qwen3.7 PlusQwen · statistically tied with #16 / 1154.5%28–79%
9GPT-6 AstraOpenAI · statistically tied with #16 / 1154.5%28–79%
9Grok 4.7SpaceXAI · statistically tied with #16 / 1154.5%28–79%
9GPT-6.1 TerraOpenAI · statistically tied with #16 / 1154.5%28–79%
9GPT-6.1 Terra ProOpenAI · statistically tied with #16 / 1154.5%28–79%
9Ministral 3 14B 2512Mistral · statistically tied with #16 / 1154.5%28–79%
9MiMo-V2.6-FlashXiaomi · statistically tied with #16 / 1154.5%28–79%
9Llama 3.3 70B InstructMeta · statistically tied with #16 / 1154.5%28–79%
9Qwen3.6 35B A3BQwen · statistically tied with #16 / 1154.5%28–79%
9Inkling SmallThinking Machines · statistically tied with #16 / 1154.5%28–79%
9Muse Spark OpenMeta · statistically tied with #16 / 1154.5%28–79%
9Gemini 3.8 FlashGoogle · statistically tied with #16 / 1154.5%28–79%
9GPT-6 LunaOpenAI · statistically tied with #16 / 1154.5%28–79%
9MiMo-V2.6-ProXiaomi · statistically tied with #16 / 1154.5%28–79%
9Kimi K3.1 ProMoonshotAI · statistically tied with #16 / 1154.5%28–79%
9Qwen3.8 Max (0902)Qwen · statistically tied with #16 / 1154.5%28–79%
28Llama 4 MaverickMeta · statistically tied with #15 / 1050.0%24–76%
29Kimi K2.7 CodeMoonshotAI · statistically tied with #15 / 1145.5%21–72%
29Muse WonderMeta · statistically tied with #15 / 1145.5%21–72%
29Ministral 3 8B 2512Mistral · statistically tied with #15 / 1145.5%21–72%
29GLM 5.3 FlashZ.ai · statistically tied with #15 / 1145.5%21–72%
29Solar Pro 4Upstage · statistically tied with #15 / 1145.5%21–72%
29Mistral Small 4Mistral · statistically tied with #15 / 1145.5%21–72%
29Qwen3 Coder NextQwen · statistically tied with #15 / 1145.5%21–72%
29Qwen3.8 27BQwen · statistically tied with #15 / 1145.5%21–72%
29GLM 5.3Z.ai · statistically tied with #15 / 1145.5%21–72%
29GPT-6.1 SolOpenAI · statistically tied with #15 / 1145.5%21–72%
29Claude Haiku 5.5Anthropic · statistically tied with #15 / 1145.5%21–72%
29GPT-6 SolOpenAI · statistically tied with #15 / 1145.5%21–72%
29MiniMax M3.1 FlashMiniMax · statistically tied with #15 / 1145.5%21–72%
42Step 5 PreviewStepFun · statistically tied with #14 / 1136.4%15–65%
42Qwen3.8 2.4T A95BQwen · statistically tied with #14 / 1136.4%15–65%
42GPT-5.6 TerraOpenAI · statistically tied with #14 / 1136.4%15–65%
42gpt-oss-20bOpenAI · statistically tied with #14 / 1136.4%15–65%
42Granite 4.0 MicroIBM · statistically tied with #14 / 1136.4%15–65%
42Ministral 3 3B 2512Mistral · statistically tied with #14 / 1136.4%15–65%
42Mercury 2.5Inception · statistically tied with #14 / 1136.4%15–65%
42Phi 4Microsoft · statistically tied with #14 / 1136.4%15–65%
42Nemotron 3.5 LightningNVIDIA · statistically tied with #14 / 1136.4%15–65%
42Hy3Tencent · statistically tied with #14 / 1136.4%15–65%
42DeepSeek V4 Flash Vision ExpDeepSeek · statistically tied with #14 / 1136.4%15–65%
42LongCat 2.0Meituan · statistically tied with #14 / 1136.4%15–65%
42Nemotron 3 UltraNVIDIA · statistically tied with #14 / 1136.4%15–65%
42DeepSeek V4 Pro 0813DeepSeek · statistically tied with #14 / 1136.4%15–65%
42DeepSeek V4.1 Flash 1017DeepSeek · statistically tied with #14 / 1136.4%15–65%
42GPT-5.4OpenAI · statistically tied with #14 / 1136.4%15–65%
42gpt-oss-120bOpenAI · statistically tied with #14 / 1136.4%15–65%
59Mistral Medium 3.5Mistral · statistically tied with #13 / 1127.3%10–57%
59Hermes 4 405BNous · statistically tied with #13 / 1127.3%10–57%
59Nova Micro 1.0Amazon · statistically tied with #13 / 1127.3%10–57%
59Trinity Large ThinkingArcee AI · statistically tied with #13 / 1127.3%10–57%
59MiniMax M3MiniMax · statistically tied with #13 / 1127.3%10–57%
64Nemotron 3.5 Super VLNVIDIA · statistically tied with #12 / 1118.2%5–48%
64Gemma 4 31BGoogle · statistically tied with #12 / 1118.2%5–48%
64Ling 3.1 FlashinclusionAI · statistically tied with #12 / 1118.2%5–48%
64Qwen3.5 397B A17BQwen · statistically tied with #12 / 1118.2%5–48%
64Solar Mini 4Upstage · statistically tied with #12 / 1118.2%5–48%
64Mistral Large 4Mistral · statistically tied with #12 / 1118.2%5–48%
70Reka Flash 3rekaai · statistically tied with #11 / 119.1%2–38%
70Gemma 4 26B A4BGoogle · statistically tied with #11 / 119.1%2–38%
70Granite 4.2 8BIBM · statistically tied with #11 / 119.1%2–38%

How hard are these questions?

Questions by the share of 72 scored models that got them right

  1. 1No model right0 questions
  2. 2Under 25% of models3 questions
  3. 325–50% of models4 questions
  4. 450–75% of models1 question
  5. 575% or more, not all3 questions
  6. 6Every model right0 questions

Science →Engineering →Economics →Law →Humanities & social science →

Medicine: frequently asked questions

Which AI model is best at medicine?
Claude Fable 5.1 and o3 and GPT-5.3-Codex and Gemini 3.5 Flash Lite and Gemini 3.1 Pro Preview and Claude Opus 5.5 and Gemini 4 Argon and Claude Sonnet 5.5 answered the most medicine questions correctly: 7 of 11. With only 11 questions, 71 other models are statistically tied with the leader, so treat this ranking as a rough guide.
Where do the medicine questions come from?
8 from SuperGPQA (ODC-BY 1.0) and 3 from MMLU-Pro (MIT). Every question is multiple choice with ten options and one correct answer.
How hard are the medicine questions?
0 of the 11 questions were answered correctly by none of the 72 scored models, and 0 by all of them.