ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

LongCat 2.0 vs Ministral 3 8B 2512: accuracy, speed and price

Similar accuracy; Ministral 3 8B 2512 costs less per question.

MeituanLongCat 2.048.8% score · #48$0.0032 per question67 tok/sMistralMinistral 3 8B 251242.5% score · #52$0.00027 per question124 tok/s

Verdict

Built from the measurements below

  • Accuracy is too close to call: LongCat 2.0 scored 48.8% and Ministral 3 8B 2512 42.5% on the same 80 questions, a gap within what chance alone could produce.
  • Ministral 3 8B 2512 is 12× cheaper per question as billed ($0.00027 vs $0.0032), reasoning tokens included.
  • Ministral 3 8B 2512 generates text 1.9× faster: 124 tokens per second against 67.
  • Ministral 3 8B 2512 starts answering sooner: 0.3 s to the first token, against 1.4 s for LongCat 2.0.
  • LongCat 2.0 accepts more context: 1M tokens against 262k.

Which should you choose?

Choose LongCat 2.0 for a 1M-token context window. Choose Ministral 3 8B 2512 for the lower cost per correct answer ($0.00063 against $0.0066), faster streaming, a quicker first token and shorter outputs (about 1,496 tokens per question against 2,600). The widest subject gap is in science: LongCat 2.0 answered 18 of 30 correctly, Ministral 3 8B 2512 10 of 30. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap +6.3 points (z = 0.79). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureLongCat 2.0Ministral 3 8B 2512Edge
Verification scoreHigher is better48.8%42.5%Too close to call
Cost per questionLower is better$0.0032$0.00027Ministral 3 8B 2512
Output speedHigher is better67124Ministral 3 8B 2512
Time to first tokenLower is better1.4 s0.3 sMinistral 3 8B 2512
End-to-end response timeLower is better10.0 s3.5 sMinistral 3 8B 2512
Tokens per questionLower is better2,6001,496Ministral 3 8B 2512
Context windowHigher is better1M262kLongCat 2.0

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

LongCat 2.0 first, Ministral 3 8B 2512 second

Verification scorehigher is better
LongCat 2.048.8%
Ministral 3 8B 251242.5%
Cost per questionlower is better
LongCat 2.0$0.0032
Ministral 3 8B 2512$0.00027
Output speedhigher is better
LongCat 2.067
Ministral 3 8B 2512124
Time to first tokenlower is better
LongCat 2.01.4 s
Ministral 3 8B 25120.3 s
End-to-end response timelower is better
LongCat 2.010.0 s
Ministral 3 8B 25123.5 s
Tokens per questionlower is better
LongCat 2.02,600
Ministral 3 8B 25121,496
Context windowhigher is better
LongCat 2.01M
Ministral 3 8B 2512262k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Mercury 2.5Phi 4Ministral 3 8B 2512Granite 4.2 8BGLM 5.3 FlashTrinity Large ThinkingMiniMax M3LongCat 2.0Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120b

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationLongCat 2.0Ministral 3 8B 2512
CreatorMeituanMistral
Released20 Jul 20262 Dec 2025
WeightsOpen weightsOpen weights
ReasoningYesNo
Inputtexttext, image
Outputtexttext
Context window1M tokens262k tokens
Knowledge cutoff——
List price in / out per 1M$0.3 / $1.2$0.15 / $0.15
Blended price per 1M (3:1)$0.52$0.15
Cheapest hostAtlasCloud · $0.3 / $1.2Mistral · $0.15 / $0.15
Hosts on OpenRouter11
Last tested26 Oct 202626 Oct 2026

Compare any two models →

LongCat 2.0 vs Ministral 3 8B 2512: frequently asked questions

Is LongCat 2.0 better than Ministral 3 8B 2512?
On accuracy they are statistically tied: LongCat 2.0 scored 48.8% and Ministral 3 8B 2512 42.5% on the same 80 questions from open datasets. Similar accuracy; Ministral 3 8B 2512 costs less per question.
Which is cheaper, LongCat 2.0 or Ministral 3 8B 2512?
Ministral 3 8B 2512 is cheaper: $0.00027 per question as billed, against $0.0032. List prices: LongCat 2.0 $0.3 in / $1.2 out, Ministral 3 8B 2512 $0.15 in / $0.15 out per million tokens.
Which is faster, LongCat 2.0 or Ministral 3 8B 2512?
Ministral 3 8B 2512 is faster at 124 output tokens per second, against 67. Time to first token: 1.4 s for LongCat 2.0, 0.3 s for Ministral 3 8B 2512.
Which has the larger context window, LongCat 2.0 or Ministral 3 8B 2512?
LongCat 2.0, with 1M tokens against 262k.