ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Phi 4 vs Solar Mini 4: accuracy, speed and price

Similar accuracy; Phi 4 costs less per question.

MicrosoftPhi 441.2% score · #54$0.00012 per question103 tok/sUpstageSolar Mini 441.2% score · #54$0.00053 per question107 tok/s

Verdict

Built from the measurements below

  • Accuracy is too close to call: Phi 4 scored 41.2% and Solar Mini 4 41.2% on the same 80 questions, a gap within what chance alone could produce.
  • Phi 4 is 4.4× cheaper per question as billed ($0.00012 vs $0.00053), reasoning tokens included.
  • Output speed is similar (103 vs 107 tokens per second).
  • Phi 4 starts answering sooner: 0.5 s to the first token, against 0.7 s for Solar Mini 4.
  • Solar Mini 4 accepts more context: 524k tokens against 16k.

Which should you choose?

Choose Phi 4 for the lower cost per correct answer ($0.00029 against $0.0013), a quicker first token, shorter outputs (about 699 tokens per question against 2,552) and open weights you can run yourself. Choose Solar Mini 4 for a 524k-token context window. The widest subject gap is in law: Phi 4 answered 2 of 6 correctly, Solar Mini 4 0 of 6. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap +0.0 points (z = 0.00). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasurePhi 4Solar Mini 4Edge
Verification scoreHigher is better41.2%41.2%Too close to call
Cost per questionLower is better$0.00012$0.00053Phi 4
Output speedHigher is better103107Similar
Time to first tokenLower is better0.5 s0.7 sPhi 4
End-to-end response timeLower is better4.9 s7.4 sPhi 4
Tokens per questionLower is better6992,552Phi 4
Context windowHigher is better16k524kSolar Mini 4

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Phi 4 first, Solar Mini 4 second

Verification scorehigher is better
Phi 441.2%
Solar Mini 441.2%
Cost per questionlower is better
Phi 4$0.00012
Solar Mini 4$0.00053
Output speedhigher is better
Phi 4103
Solar Mini 4107
Time to first tokenlower is better
Phi 40.5 s
Solar Mini 40.7 s
End-to-end response timelower is better
Phi 44.9 s
Solar Mini 47.4 s
Tokens per questionlower is better
Phi 4699
Solar Mini 42,552
Context windowhigher is better
Phi 416k
Solar Mini 4524k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationPhi 4Solar Mini 4
CreatorMicrosoftUpstage
Released10 Jan 202523 Sept 2026
WeightsOpen weights—
ReasoningNoYes
Inputtexttext
Outputtexttext
Context window16k tokens524k tokens
Knowledge cutoff30 Jun 2024—
List price in / out per 1M$0.07 / $0.14$0.05 / $0.2
Blended price per 1M (3:1)$0.09$0.09
Cheapest hostDeepInfra · $0.056 / $0.112Upstage · $0.05 / $0.2
Hosts on OpenRouter11
Last tested26 Oct 202626 Oct 2026

Compare any two models →

Phi 4 vs Solar Mini 4: frequently asked questions

Is Phi 4 better than Solar Mini 4?
On accuracy they are statistically tied: Phi 4 scored 41.2% and Solar Mini 4 41.2% on the same 80 questions from open datasets. Similar accuracy; Phi 4 costs less per question.
Which is cheaper, Phi 4 or Solar Mini 4?
Phi 4 is cheaper: $0.00012 per question as billed, against $0.00053. List prices: Phi 4 $0.07 in / $0.14 out, Solar Mini 4 $0.05 in / $0.2 out per million tokens.
Which is faster, Phi 4 or Solar Mini 4?
Their output speed is similar: 103 and 107 tokens per second.
Which has the larger context window, Phi 4 or Solar Mini 4?
Solar Mini 4, with 524k tokens against 16k.