Is Hermes 4 405B better than Mistral Medium 3.5?

On accuracy they are statistically tied: Hermes 4 405B scored 26.2% and Mistral Medium 3.5 28.8% on the same 80 questions from open datasets. Similar accuracy; Hermes 4 405B costs less per question.

Which is cheaper, Hermes 4 405B or Mistral Medium 3.5?

Hermes 4 405B is cheaper: $0.01 per question as billed, against $0.025. List prices: Hermes 4 405B $1 in / $3 out, Mistral Medium 3.5 $1.5 in / $7.5 out per million tokens.

Which is faster, Hermes 4 405B or Mistral Medium 3.5?

Mistral Medium 3.5 is faster at 145 output tokens per second, against 41. Time to first token: 0.3 s for Hermes 4 405B, 0.5 s for Mistral Medium 3.5.

Which has the larger context window, Hermes 4 405B or Mistral Medium 3.5?

Mistral Medium 3.5, with 262k tokens against 131k.

Head to head · tested 27 Oct 2026

Hermes 4 405B vs Mistral Medium 3.5: accuracy, speed and price

Similar accuracy; Hermes 4 405B costs less per question.

NousHermes 4 405B26.2% score · #67$0.01 per question41 tok/s

MistralMistral Medium 3.528.8% score · #65$0.025 per question145 tok/s

Verdict

Built from the measurements below

Accuracy is too close to call: Hermes 4 405B scored 26.2% and Mistral Medium 3.5 28.8% on the same 80 questions, a gap within what chance alone could produce.
Hermes 4 405B is 2.5× cheaper per question as billed ($0.01 vs $0.025), reasoning tokens included.
Mistral Medium 3.5 generates text 3.6× faster: 145 tokens per second against 41.
Hermes 4 405B starts answering sooner: 0.3 s to the first token, against 0.5 s for Mistral Medium 3.5.
Mistral Medium 3.5 accepts more context: 262k tokens against 131k.

Which should you choose?

Choose Hermes 4 405B for the lower cost per correct answer ($0.038 against $0.087), a quicker first token and open weights you can run yourself. Choose Mistral Medium 3.5 for faster streaming and a 262k-token context window.

Accuracy gap -2.5 points (z = -0.35). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

Measure	Hermes 4 405B	Mistral Medium 3.5	Edge
Verification scoreHigher is better	26.2%	28.8%	Too close to call
Cost per questionLower is better	$0.01	$0.025	Hermes 4 405B
Output speedHigher is better	41	145	Mistral Medium 3.5
Time to first tokenLower is better	0.3 s	0.5 s	Hermes 4 405B
End-to-end response timeLower is better	17.5 s	5.4 s	Mistral Medium 3.5
Tokens per questionLower is better	3,223	3,268	Similar
Context windowHigher is better	131k	262k	Mistral Medium 3.5

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Hermes 4 405B first, Mistral Medium 3.5 second

Verification scorehigher is better

Hermes 4 405B26.2%

Mistral Medium 3.528.8%

Cost per questionlower is better

Hermes 4 405B$0.01

Mistral Medium 3.5$0.025

Output speedhigher is better

Hermes 4 405B41

Mistral Medium 3.5145

Time to first tokenlower is better

Hermes 4 405B0.3 s

Mistral Medium 3.50.5 s

End-to-end response timelower is better

Hermes 4 405B17.5 s

Mistral Medium 3.55.4 s

Tokens per questionlower is better

Hermes 4 405B3,223

Mistral Medium 3.53,268

Context windowhigher is better

Hermes 4 405B131k

Mistral Medium 3.5262k

Score vs. cost per question

Both models in color, every other model in grey

Results by subject and dataset

Questions answered correctly. Small samples: a gap of one or two questions is noise.

Subject	Questions	Hermes 4 405B	Mistral Medium 3.5
Science	30	5/30	5/30
Engineering	15	0/15	0/15
Medicine	11	3/11	3/11
Economics	9	5/9	6/9
Law	6	2/6	2/6
Humanities & social science	9	6/9	7/9
SuperGPQA dataset	50	3/50	5/50
MMLU-Pro dataset	30	18/30	18/30

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

Specification	Hermes 4 405B	Mistral Medium 3.5
Creator	Nous	Mistral
Released	26 Aug 2025	30 Apr 2026
Weights	Open weights	—
Reasoning	Yes	Yes
Input	text	text, image, file
Output	text	text
Context window	131k tokens	262k tokens
Knowledge cutoff	31 Aug 2024	—
List price in / out per 1M	$1 / $3	$1.5 / $7.5
Blended price per 1M (3:1)	$1.50	$3.00
Cheapest host	Nebius · $1 / $3	Mistral · $1.5 / $7.5
Hosts on OpenRouter	1	1
Last tested	27 Oct 2026	27 Oct 2026

Compare any two models →

Hermes 4 405B vs Mistral Medium 3.5: frequently asked questions

Is Hermes 4 405B better than Mistral Medium 3.5?: On accuracy they are statistically tied: Hermes 4 405B scored 26.2% and Mistral Medium 3.5 28.8% on the same 80 questions from open datasets. Similar accuracy; Hermes 4 405B costs less per question.
Which is cheaper, Hermes 4 405B or Mistral Medium 3.5?: Hermes 4 405B is cheaper: $0.01 per question as billed, against $0.025. List prices: Hermes 4 405B $1 in / $3 out, Mistral Medium 3.5 $1.5 in / $7.5 out per million tokens.
Which is faster, Hermes 4 405B or Mistral Medium 3.5?: Mistral Medium 3.5 is faster at 145 output tokens per second, against 41. Time to first token: 0.3 s for Hermes 4 405B, 0.5 s for Mistral Medium 3.5.
Which has the larger context window, Hermes 4 405B or Mistral Medium 3.5?: Mistral Medium 3.5, with 262k tokens against 131k.