ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Provider

Mistral AI models

6 models tested, compared with 72 models from other labs.

Best verification score50.0Mistral Large 4 · #47 of 78Best output speed252Ministral 3 3B 2512 · #4 of 78Best time to first token0.3 sMinistral 3 3B 2512 · #1 of 78Best cost per question$0.00019Ministral 3 3B 2512 · #9 of 78
Mistral#47Mistral Large 450.0% verification score83 tok/s1.1 s first token$0.011 / questionMistral#52Ministral 3 8B 251242.5% verification score124 tok/s0.3 s first token$0.00027 / questionMistral#61Ministral 3 14B 251233.8% verification score32 tok/s0.6 s first token$0.00031 / questionMistral#62Mistral Small 432.5% verification score170 tok/s0.5 s first token$0.0019 / questionMistral#65Mistral Medium 3.528.8% verification score145 tok/s0.5 s first token$0.025 / questionMistral#67Ministral 3 3B 251226.2% verification score252 tok/s0.3 s first token$0.00019 / question

Score vs. cost per question

Mistral models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bGranite 4.0 MicroMinistral 3 3B 2512Nova Micro 1.0Phi 4Ministral 3 8B 2512Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4MiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Output speed vs. cost per question

Mistral models in color, every other model in grey

Most attractive quadrant0200400600800$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Output speed (tok/s) →Nemotron 3.5 Super VLQwen3.5 397B A17BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Trinity Large ThinkingInkling SmallGemini 3.5 Flash LiteMuse Spark OpenGPT-6 Luna

Verification score

Mistral models highlighted against every model in this run · higher is better

  1. 1Claude Opus 5.585.0
  2. 2Gemini 4 Argon83.8
  3. 3GPT-5.3-Codex82.5
  4. 3Gemini 3.8 Flash82.5
  5. 5Claude Fable 5.181.2
  6. 6Grok 4.780.0
  7. 7Muse Wonder78.8
  8. 7Claude Sonnet 5.578.8
  9. 9Gemini 3.1 Pro Preview77.5
  10. 9Muse Spark Open77.5
  11. 47Mistral Large 450.0
  12. 52Ministral 3 8B 251242.5
  13. 61Ministral 3 14B 251233.8
  14. 62Mistral Small 432.5
  15. 65Mistral Medium 3.528.8
  16. 67Ministral 3 3B 251226.2

56 more models · see the full leaderboard

6 of 78 tested models have no verification score

Output speed

Mistral models highlighted against every model in this run · higher is better

  1. 1Nemotron 3.5 Super VL715
  2. 2Gemini 3.5 Flash Lite409
  3. 3Trinity Large Thinking329
  4. 4Ministral 3 3B 2512252
  5. 5Qwen3.5 397B A17B240
  6. 6Inkling Small233
  7. 7Muse Spark Open220
  8. 8Claude Haiku 5.5192
  9. 9Nova Micro 1.0180
  10. 10Muse Wonder175
  11. 12Mistral Small 4170
  12. 17Mistral Medium 3.5145
  13. 27Ministral 3 8B 2512124
  14. 44Mistral Large 483
  15. 68Ministral 3 14B 251232

57 more models · see the full leaderboard

6 of 78 tested models have no output speed

Time to first token

Mistral models highlighted against every model in this run · lower is better

  1. 1Ministral 3 3B 25120.3 s
  2. 2Ministral 3 8B 25120.3 s
  3. 3Hermes 4 405B0.3 s
  4. 4Kimi K2.7 Code0.4 s
  5. 5MiMo-V2.6-Flash0.4 s
  6. 6Qwen3.6 35B A3B0.4 s
  7. 7Qwen3 Coder Next0.4 s
  8. 8Mistral Medium 3.50.5 s
  9. 9Phi 40.5 s
  10. 10Nova Micro 1.00.5 s
  11. 12Mistral Small 40.5 s
  12. 29Ministral 3 14B 25120.6 s
  13. 45Mistral Large 41.1 s

61 more models · see the full leaderboard

4 of 78 tested models have no time to first token

Cost per question

Mistral models highlighted against every model in this run · lower is better

  1. 1Granite 4.0 Micro$0.000051
  2. 2gpt-oss-20b$0.000066
  3. 3Granite 4.2 8B$0.000077
  4. 4Nova Micro 1.0$0.000087
  5. 5Mercury 2.5$0.00011
  6. 6Phi 4$0.00012
  7. 7gpt-oss-120b$0.00013
  8. 8GLM 5.3 Flash$0.00015
  9. 9Ministral 3 3B 2512$0.00019
  10. 10GPT-6 Luna$0.00025
  11. 11Ministral 3 8B 2512$0.00027
  12. 12Ministral 3 14B 2512$0.00031
  13. 35Mistral Small 4$0.0019
  14. 62Mistral Large 4$0.011
  15. 71Mistral Medium 3.5$0.025

57 more models · see the full leaderboard

6 of 78 tested models have no cost per question
  • LFM2.5-2.6B: no verification score (80 of 80 questions failed on the host's side)
  • Ling 3.0 Flash VL: no verification score (not yet run on the current question suite)
  • Qwen3.5-122B-A10B: no verification score (not yet run on the current question suite)
  • Qwen3 Next 80B A3B Thinking: no verification score (not yet run on the current question suite)
  • Nova Premier 1.0: no verification score (not yet run on the current question suite)
  • GPT-5.5 Pro: no verification score (not yet run on the current question suite)

End-to-end response time

Mistral models highlighted against every model in this run · lower is better

  1. 1Nemotron 3.5 Super VL1.5 s
  2. 2Ministral 3 3B 25121.6 s
  3. 3Nova Micro 1.02.1 s
  4. 4Trinity Large Thinking2.7 s
  5. 5Gemini 3.5 Flash Lite2.8 s
  6. 6DeepSeek V4 Flash Vision Exp3.2 s
  7. 7Muse Spark Open3.2 s
  8. 8GLM 5.33.4 s
  9. 9Ministral 3 8B 25123.5 s
  10. 10Qwen3.5 397B A17B3.5 s
  11. 21Mistral Small 44.5 s
  12. 28Mistral Medium 3.55.4 s
  13. 54Mistral Large 49.5 s
  14. 64Ministral 3 14B 251212.7 s

60 more models · see the full leaderboard

4 of 78 tested models have no end-to-end response time

All Mistral results

Leaderboard
#
1Mistral Large 4
Mistral · served by Mistral
50.040 / 80831.1 s$0.011$1.36 / $4.181M
2Ministral 3 8B 2512
Mistral · served by Mistral
42.534 / 801240.3 s$0.00027$0.15 / $0.15262k
3Ministral 3 14B 2512
Mistral · served by Mistral
33.827 / 80320.6 s$0.00031$0.2 / $0.2262k
4Mistral Small 4
Mistral · served by Mistral
32.526 / 801700.5 s$0.0019$0.15 / $0.6262k
5Mistral Medium 3.5
Mistral · served by Mistral
28.823 / 801450.5 s$0.025$1.5 / $7.5262k
6Ministral 3 3B 2512
Mistral · served by Mistral
26.221 / 802520.3 s$0.00019$0.1 / $0.1131k

Mistral: quick answers

What is the best Mistral model?
Mistral Large 4 has the highest verification score of the Mistral models the leaderboard tested: 50.0% (40 of 80 correct) in the 2026-10-27 run.
What is the fastest Mistral model?
Ministral 3 3B 2512 is the fastest Mistral model the leaderboard tested, at a median 252 output tokens per second.
What is the cheapest Mistral model?
Ministral 3 3B 2512 has the lowest billed cost per question of the Mistral models the leaderboard tested: $0.00019.
Which Mistral models do you test?
Mistral Large 4, Ministral 3 8B 2512, Ministral 3 14B 2512, Mistral Small 4, Mistral Medium 3.5, Ministral 3 3B 2512.