ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Qwen3.8 2.4T A95B vs Step 5 Preview: accuracy, speed and price

Similar accuracy; Step 5 Preview costs less per question.

QwenQwen3.8 2.4T A95B57.5% score · #43$0.015 per question44 tok/sStepFunStep 5 Preview63.8% score · #34$0.0047 per question93 tok/s

Verdict

Built from the measurements below

  • Accuracy is too close to call: Qwen3.8 2.4T A95B scored 57.5% and Step 5 Preview 63.8% on the same 80 questions, a gap within what chance alone could produce.
  • Step 5 Preview is 3.2× cheaper per question as billed ($0.0047 vs $0.015), reasoning tokens included.
  • Step 5 Preview generates text 2.1× faster: 93 tokens per second against 44.
  • Step 5 Preview starts answering sooner: 0.6 s to the first token, against 3.1 s for Qwen3.8 2.4T A95B.

Which should you choose?

Choose Step 5 Preview for the lower cost per correct answer ($0.0074 against $0.026), faster streaming and a quicker first token.

Accuracy gap -6.3 points (z = -0.81). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureQwen3.8 2.4T A95BStep 5 PreviewEdge
Verification scoreHigher is better57.5%63.8%Too close to call
Cost per questionLower is better$0.015$0.0047Step 5 Preview
Output speedHigher is better4493Step 5 Preview
Time to first tokenLower is better3.1 s0.6 sStep 5 Preview
End-to-end response timeLower is better14.8 s8.2 sStep 5 Preview
Tokens per questionLower is better2,4081,630Step 5 Preview
Context windowHigher is better1M1MSimilar

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Qwen3.8 2.4T A95B first, Step 5 Preview second

Verification scorehigher is better
Qwen3.8 2.4T A95B57.5%
Step 5 Preview63.8%
Cost per questionlower is better
Qwen3.8 2.4T A95B$0.015
Step 5 Preview$0.0047
Output speedhigher is better
Qwen3.8 2.4T A95B44
Step 5 Preview93
Time to first tokenlower is better
Qwen3.8 2.4T A95B3.1 s
Step 5 Preview0.6 s
End-to-end response timelower is better
Qwen3.8 2.4T A95B14.8 s
Step 5 Preview8.2 s
Tokens per questionlower is better
Qwen3.8 2.4T A95B2,408
Step 5 Preview1,630
Context windowhigher is better
Qwen3.8 2.4T A95B1M
Step 5 Preview1M

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationQwen3.8 2.4T A95BStep 5 Preview
CreatorQwenStepFun
Released12 Aug 20268 Oct 2026
WeightsOpen weightsOpen weights
ReasoningYesYes
Inputtexttext, image, video
Outputtexttext
Context window1M tokens1M tokens
Knowledge cutoff——
List price in / out per 1M$2 / $6$1 / $2.7
Blended price per 1M (3:1)$3.00$1.43
Cheapest hostNovita · $2 / $6InferenceNet · $1 / $2.7
Hosts on OpenRouter54
Last tested27 Oct 202627 Oct 2026

Compare any two models →

Qwen3.8 2.4T A95B vs Step 5 Preview: frequently asked questions

Is Qwen3.8 2.4T A95B better than Step 5 Preview?
On accuracy they are statistically tied: Qwen3.8 2.4T A95B scored 57.5% and Step 5 Preview 63.8% on the same 80 questions from open datasets. Similar accuracy; Step 5 Preview costs less per question.
Which is cheaper, Qwen3.8 2.4T A95B or Step 5 Preview?
Step 5 Preview is cheaper: $0.0047 per question as billed, against $0.015. List prices: Qwen3.8 2.4T A95B $2 in / $6 out, Step 5 Preview $1 in / $2.7 out per million tokens.
Which is faster, Qwen3.8 2.4T A95B or Step 5 Preview?
Step 5 Preview is faster at 93 output tokens per second, against 44. Time to first token: 3.1 s for Qwen3.8 2.4T A95B, 0.6 s for Step 5 Preview.
Which has the larger context window, Qwen3.8 2.4T A95B or Step 5 Preview?
Both accept about the same amount of context: 1M and 1M tokens.