ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Nemotron 3.5 Super VL vs Nemotron 3 Ultra: accuracy, speed and price

Nemotron 3 Ultra is more accurate; Nemotron 3.5 Super VL is cheaper.

NVIDIANemotron 3.5 Super VL40.0% score · #58$0.0013 per question715 tok/sNVIDIANemotron 3 Ultra60.0% score · #38$0.0057 per question136 tok/s

Verdict

Built from the measurements below

  • Nemotron 3 Ultra is more accurate: 60.0% against 40.0% on the same 80 questions, a 20.0-point gap that is statistically significant.
  • Nemotron 3.5 Super VL is 4.5× cheaper per question as billed ($0.0013 vs $0.0057), reasoning tokens included.
  • Nemotron 3.5 Super VL generates text 5.3× faster: 715 tokens per second against 136.
  • Nemotron 3.5 Super VL starts answering sooner: 0.6 s to the first token, against 0.8 s for Nemotron 3 Ultra.
  • Nemotron 3.5 Super VL accepts more context: 1M tokens against 262k.

Which should you choose?

Choose Nemotron 3.5 Super VL for the lower cost per correct answer ($0.0032 against $0.0094), faster streaming, a quicker first token, a 1M-token context window and more hosts to choose from on OpenRouter (9 against 3). Choose Nemotron 3 Ultra for the most accurate answers of the two. The widest subject gap is in science: Nemotron 3 Ultra answered 22 of 30 correctly, Nemotron 3.5 Super VL 13 of 30. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap -20.0 points (z = -2.53). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureNemotron 3.5 Super VLNemotron 3 UltraEdge
Verification scoreHigher is better40.0%60.0%Nemotron 3 Ultra
Cost per questionLower is better$0.0013$0.0057Nemotron 3.5 Super VL
Output speedHigher is better715136Nemotron 3.5 Super VL
Time to first tokenLower is better0.6 s0.8 sNemotron 3.5 Super VL
End-to-end response timeLower is better1.5 s3.6 sNemotron 3.5 Super VL
Tokens per questionLower is better2,7712,177Nemotron 3 Ultra
Context windowHigher is better1M262kNemotron 3.5 Super VL

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Nemotron 3.5 Super VL first, Nemotron 3 Ultra second

Verification scorehigher is better
Nemotron 3.5 Super VL40.0%
Nemotron 3 Ultra60.0%
Cost per questionlower is better
Nemotron 3.5 Super VL$0.0013
Nemotron 3 Ultra$0.0057
Output speedhigher is better
Nemotron 3.5 Super VL715
Nemotron 3 Ultra136
Time to first tokenlower is better
Nemotron 3.5 Super VL0.6 s
Nemotron 3 Ultra0.8 s
End-to-end response timelower is better
Nemotron 3.5 Super VL1.5 s
Nemotron 3 Ultra3.6 s
Tokens per questionlower is better
Nemotron 3.5 Super VL2,771
Nemotron 3 Ultra2,177
Context windowhigher is better
Nemotron 3.5 Super VL1M
Nemotron 3 Ultra262k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationNemotron 3.5 Super VLNemotron 3 Ultra
CreatorNVIDIANVIDIA
Released15 Oct 20264 Jun 2026
WeightsOpen weightsOpen weights
ReasoningYesYes
Inputtext, image, videotext
Outputtexttext
Context window1M tokens262k tokens
Knowledge cutoff——
List price in / out per 1M$0.15 / $0.5$0.6 / $2.4
Blended price per 1M (3:1)$0.24$1.05
Cheapest hostDarkbloom · $0.06 / $0.35DeepInfra · $0.4 / $1.76
Hosts on OpenRouter93
Last tested27 Oct 202626 Oct 2026

Compare any two models →

Nemotron 3.5 Super VL vs Nemotron 3 Ultra: frequently asked questions

Is Nemotron 3.5 Super VL better than Nemotron 3 Ultra?
Nemotron 3 Ultra is more accurate: it answered 48 of 80 questions correctly (60.0%) against 32 of 80 (40.0%) for Nemotron 3.5 Super VL. Nemotron 3 Ultra is more accurate; Nemotron 3.5 Super VL is cheaper.
Which is cheaper, Nemotron 3.5 Super VL or Nemotron 3 Ultra?
Nemotron 3.5 Super VL is cheaper: $0.0013 per question as billed, against $0.0057. List prices: Nemotron 3.5 Super VL $0.15 in / $0.5 out, Nemotron 3 Ultra $0.6 in / $2.4 out per million tokens.
Which is faster, Nemotron 3.5 Super VL or Nemotron 3 Ultra?
Nemotron 3.5 Super VL is faster at 715 output tokens per second, against 136. Time to first token: 0.6 s for Nemotron 3.5 Super VL, 0.8 s for Nemotron 3 Ultra.
Which has the larger context window, Nemotron 3.5 Super VL or Nemotron 3 Ultra?
Nemotron 3.5 Super VL, with 1M tokens against 262k.