ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Nemotron 3 Ultra vs Qwen3.8 2.4T A95B: accuracy, speed and price

Similar accuracy; Nemotron 3 Ultra costs less per question.

NVIDIANemotron 3 Ultra60.0% score · #38$0.0057 per question136 tok/sQwenQwen3.8 2.4T A95B57.5% score · #43$0.015 per question44 tok/s

Verdict

Built from the measurements below

  • Accuracy is too close to call: Nemotron 3 Ultra scored 60.0% and Qwen3.8 2.4T A95B 57.5% on the same 80 questions, a gap within what chance alone could produce.
  • Nemotron 3 Ultra is 2.7× cheaper per question as billed ($0.0057 vs $0.015), reasoning tokens included.
  • Nemotron 3 Ultra generates text 3.1× faster: 136 tokens per second against 44.
  • Nemotron 3 Ultra starts answering sooner: 0.8 s to the first token, against 3.1 s for Qwen3.8 2.4T A95B.
  • Qwen3.8 2.4T A95B accepts more context: 1M tokens against 262k.

Which should you choose?

Choose Nemotron 3 Ultra for the lower cost per correct answer ($0.0094 against $0.026), faster streaming and a quicker first token. Choose Qwen3.8 2.4T A95B for a 1M-token context window and more hosts to choose from on OpenRouter (5 against 3). The widest subject gap is in humanities & social science: Nemotron 3 Ultra answered 9 of 9 correctly, Qwen3.8 2.4T A95B 7 of 9. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap +2.5 points (z = 0.32). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureNemotron 3 UltraQwen3.8 2.4T A95BEdge
Verification scoreHigher is better60.0%57.5%Too close to call
Cost per questionLower is better$0.0057$0.015Nemotron 3 Ultra
Output speedHigher is better13644Nemotron 3 Ultra
Time to first tokenLower is better0.8 s3.1 sNemotron 3 Ultra
End-to-end response timeLower is better3.6 s14.8 sNemotron 3 Ultra
Tokens per questionLower is better2,1772,408Nemotron 3 Ultra
Context windowHigher is better262k1MQwen3.8 2.4T A95B

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Nemotron 3 Ultra first, Qwen3.8 2.4T A95B second

Verification scorehigher is better
Nemotron 3 Ultra60.0%
Qwen3.8 2.4T A95B57.5%
Cost per questionlower is better
Nemotron 3 Ultra$0.0057
Qwen3.8 2.4T A95B$0.015
Output speedhigher is better
Nemotron 3 Ultra136
Qwen3.8 2.4T A95B44
Time to first tokenlower is better
Nemotron 3 Ultra0.8 s
Qwen3.8 2.4T A95B3.1 s
End-to-end response timelower is better
Nemotron 3 Ultra3.6 s
Qwen3.8 2.4T A95B14.8 s
Tokens per questionlower is better
Nemotron 3 Ultra2,177
Qwen3.8 2.4T A95B2,408
Context windowhigher is better
Nemotron 3 Ultra262k
Qwen3.8 2.4T A95B1M

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationNemotron 3 UltraQwen3.8 2.4T A95B
CreatorNVIDIAQwen
Released4 Jun 202612 Aug 2026
WeightsOpen weightsOpen weights
ReasoningYesYes
Inputtexttext
Outputtexttext
Context window262k tokens1M tokens
Knowledge cutoff——
List price in / out per 1M$0.6 / $2.4$2 / $6
Blended price per 1M (3:1)$1.05$3.00
Cheapest hostDeepInfra · $0.4 / $1.76Novita · $2 / $6
Hosts on OpenRouter35
Last tested26 Oct 202627 Oct 2026

Compare any two models →

Nemotron 3 Ultra vs Qwen3.8 2.4T A95B: frequently asked questions

Is Nemotron 3 Ultra better than Qwen3.8 2.4T A95B?
On accuracy they are statistically tied: Nemotron 3 Ultra scored 60.0% and Qwen3.8 2.4T A95B 57.5% on the same 80 questions from open datasets. Similar accuracy; Nemotron 3 Ultra costs less per question.
Which is cheaper, Nemotron 3 Ultra or Qwen3.8 2.4T A95B?
Nemotron 3 Ultra is cheaper: $0.0057 per question as billed, against $0.015. List prices: Nemotron 3 Ultra $0.6 in / $2.4 out, Qwen3.8 2.4T A95B $2 in / $6 out per million tokens.
Which is faster, Nemotron 3 Ultra or Qwen3.8 2.4T A95B?
Nemotron 3 Ultra is faster at 136 output tokens per second, against 44. Time to first token: 0.8 s for Nemotron 3 Ultra, 3.1 s for Qwen3.8 2.4T A95B.
Which has the larger context window, Nemotron 3 Ultra or Qwen3.8 2.4T A95B?
Qwen3.8 2.4T A95B, with 1M tokens against 262k.