ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Mistral Large 4 vs Mistral Medium 3.5: accuracy, speed and price

Mistral Large 4 is more accurate and cheaper.

MistralMistral Large 450.0% score · #47$0.011 per question83 tok/sMistralMistral Medium 3.528.8% score · #65$0.025 per question145 tok/s

Verdict

Built from the measurements below

  • Mistral Large 4 is more accurate: 50.0% against 28.8% on the same 80 questions, a 21.3-point gap that is statistically significant.
  • Mistral Large 4 is 2.2× cheaper per question as billed ($0.011 vs $0.025), reasoning tokens included.
  • Mistral Medium 3.5 generates text 1.7× faster: 145 tokens per second against 83.
  • Mistral Medium 3.5 starts answering sooner: 0.5 s to the first token, against 1.1 s for Mistral Large 4.
  • Mistral Large 4 accepts more context: 1M tokens against 262k.

Which should you choose?

Choose Mistral Large 4 for the most accurate answers of the two, the lower cost per correct answer ($0.023 against $0.087) and a 1M-token context window. Choose Mistral Medium 3.5 for faster streaming and a quicker first token. The widest subject gap is in science: Mistral Large 4 answered 20 of 30 correctly, Mistral Medium 3.5 5 of 30. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap +21.3 points (z = 2.75). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureMistral Large 4Mistral Medium 3.5Edge
Verification scoreHigher is better50.0%28.8%Mistral Large 4
Cost per questionLower is better$0.011$0.025Mistral Large 4
Output speedHigher is better83145Mistral Medium 3.5
Time to first tokenLower is better1.1 s0.5 sMistral Medium 3.5
End-to-end response timeLower is better9.5 s5.4 sMistral Medium 3.5
Tokens per questionLower is better2,6133,268Mistral Large 4
Context windowHigher is better1M262kMistral Large 4

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Mistral Large 4 first, Mistral Medium 3.5 second

Verification scorehigher is better
Mistral Large 450.0%
Mistral Medium 3.528.8%
Cost per questionlower is better
Mistral Large 4$0.011
Mistral Medium 3.5$0.025
Output speedhigher is better
Mistral Large 483
Mistral Medium 3.5145
Time to first tokenlower is better
Mistral Large 41.1 s
Mistral Medium 3.50.5 s
End-to-end response timelower is better
Mistral Large 49.5 s
Mistral Medium 3.55.4 s
Tokens per questionlower is better
Mistral Large 42,613
Mistral Medium 3.53,268
Context windowhigher is better
Mistral Large 41M
Mistral Medium 3.5262k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationMistral Large 4Mistral Medium 3.5
CreatorMistralMistral
Released6 Oct 202630 Apr 2026
Weights——
ReasoningYesYes
Inputtext, imagetext, image, file
Outputtexttext
Context window1M tokens262k tokens
Knowledge cutoff——
List price in / out per 1M$1.36 / $4.18$1.5 / $7.5
Blended price per 1M (3:1)$2.06$3.00
Cheapest hostMistral · $1.36 / $4.18Mistral · $1.5 / $7.5
Hosts on OpenRouter11
Last tested26 Oct 202627 Oct 2026

Compare any two models →

Mistral Large 4 vs Mistral Medium 3.5: frequently asked questions

Is Mistral Large 4 better than Mistral Medium 3.5?
Mistral Large 4 is more accurate: it answered 40 of 80 questions correctly (50.0%) against 23 of 80 (28.8%) for Mistral Medium 3.5. Mistral Large 4 is more accurate and cheaper.
Which is cheaper, Mistral Large 4 or Mistral Medium 3.5?
Mistral Large 4 is cheaper: $0.011 per question as billed, against $0.025. List prices: Mistral Large 4 $1.36 in / $4.18 out, Mistral Medium 3.5 $1.5 in / $7.5 out per million tokens.
Which is faster, Mistral Large 4 or Mistral Medium 3.5?
Mistral Medium 3.5 is faster at 145 output tokens per second, against 83. Time to first token: 1.1 s for Mistral Large 4, 0.5 s for Mistral Medium 3.5.
Which has the larger context window, Mistral Large 4 or Mistral Medium 3.5?
Mistral Large 4, with 1M tokens against 262k.