ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Provider

NVIDIA AI models

3 models tested, compared with 75 models from other labs.

Best verification score60.0Nemotron 3 Ultra · #38 of 78Best output speed715Nemotron 3.5 Super VL · #1 of 78Best time to first token0.5 sNemotron 3.5 Lightning · #13 of 78Best cost per question$0.00049Nemotron 3.5 Lightning · #16 of 78
NVIDIA#38Nemotron 3 Ultra60.0% verification score136 tok/s0.8 s first token$0.0057 / questionNVIDIA#51Nemotron 3.5 Lightning43.8% verification score116 tok/s0.5 s first token$0.00049 / questionNVIDIA#58Nemotron 3.5 Super VL40.0% verification score715 tok/s0.6 s first token$0.0013 / question

Score vs. cost per question

NVIDIA models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Mercury 2.5Phi 4Nemotron 3.5 LightningGranite 4.2 8BHy3GLM 5.3 FlashTrinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120b

Output speed vs. cost per question

NVIDIA models in color, every other model in grey

Most attractive quadrant0200400600800$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Output speed (tok/s) →Nemotron 3.5 Super VLQwen3.5 397B A17BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Trinity Large ThinkingInkling SmallGemini 3.5 Flash LiteMuse Spark OpenGPT-6 Luna

Verification score

NVIDIA models highlighted against every model in this run · higher is better

  1. 1Claude Opus 5.585.0
  2. 2Gemini 4 Argon83.8
  3. 3GPT-5.3-Codex82.5
  4. 3Gemini 3.8 Flash82.5
  5. 5Claude Fable 5.181.2
  6. 6Grok 4.780.0
  7. 7Muse Wonder78.8
  8. 7Claude Sonnet 5.578.8
  9. 9Gemini 3.1 Pro Preview77.5
  10. 9Muse Spark Open77.5
  11. 38Nemotron 3 Ultra60.0
  12. 51Nemotron 3.5 Lightning43.8
  13. 58Nemotron 3.5 Super VL40.0

59 more models · see the full leaderboard

6 of 78 tested models have no verification score

Output speed

NVIDIA models highlighted against every model in this run · higher is better

  1. 1Nemotron 3.5 Super VL715
  2. 2Gemini 3.5 Flash Lite409
  3. 3Trinity Large Thinking329
  4. 4Ministral 3 3B 2512252
  5. 5Qwen3.5 397B A17B240
  6. 6Inkling Small233
  7. 7Muse Spark Open220
  8. 8Claude Haiku 5.5192
  9. 9Nova Micro 1.0180
  10. 10Muse Wonder175
  11. 21Nemotron 3 Ultra136
  12. 31Nemotron 3.5 Lightning116

60 more models · see the full leaderboard

6 of 78 tested models have no output speed

Time to first token

NVIDIA models highlighted against every model in this run · lower is better

  1. 1Ministral 3 3B 25120.3 s
  2. 2Ministral 3 8B 25120.3 s
  3. 3Hermes 4 405B0.3 s
  4. 4Kimi K2.7 Code0.4 s
  5. 5MiMo-V2.6-Flash0.4 s
  6. 6Qwen3.6 35B A3B0.4 s
  7. 7Qwen3 Coder Next0.4 s
  8. 8Mistral Medium 3.50.5 s
  9. 9Phi 40.5 s
  10. 10Nova Micro 1.00.5 s
  11. 13Nemotron 3.5 Lightning0.5 s
  12. 28Nemotron 3.5 Super VL0.6 s
  13. 41Nemotron 3 Ultra0.8 s

61 more models · see the full leaderboard

4 of 78 tested models have no time to first token

Cost per question

NVIDIA models highlighted against every model in this run · lower is better

  1. 1Granite 4.0 Micro$0.000051
  2. 2gpt-oss-20b$0.000066
  3. 3Granite 4.2 8B$0.000077
  4. 4Nova Micro 1.0$0.000087
  5. 5Mercury 2.5$0.00011
  6. 6Phi 4$0.00012
  7. 7gpt-oss-120b$0.00013
  8. 8GLM 5.3 Flash$0.00015
  9. 9Ministral 3 3B 2512$0.00019
  10. 10GPT-6 Luna$0.00025
  11. 16Nemotron 3.5 Lightning$0.00049
  12. 30Nemotron 3.5 Super VL$0.0013
  13. 52Nemotron 3 Ultra$0.0057

59 more models · see the full leaderboard

6 of 78 tested models have no cost per question
  • LFM2.5-2.6B: no verification score (80 of 80 questions failed on the host's side)
  • Ling 3.0 Flash VL: no verification score (not yet run on the current question suite)
  • Qwen3.5-122B-A10B: no verification score (not yet run on the current question suite)
  • Qwen3 Next 80B A3B Thinking: no verification score (not yet run on the current question suite)
  • Nova Premier 1.0: no verification score (not yet run on the current question suite)
  • GPT-5.5 Pro: no verification score (not yet run on the current question suite)

End-to-end response time

NVIDIA models highlighted against every model in this run · lower is better

  1. 1Nemotron 3.5 Super VL1.5 s
  2. 2Ministral 3 3B 25121.6 s
  3. 3Nova Micro 1.02.1 s
  4. 4Trinity Large Thinking2.7 s
  5. 5Gemini 3.5 Flash Lite2.8 s
  6. 6DeepSeek V4 Flash Vision Exp3.2 s
  7. 7Muse Spark Open3.2 s
  8. 8GLM 5.33.4 s
  9. 9Ministral 3 8B 25123.5 s
  10. 10Qwen3.5 397B A17B3.5 s
  11. 12Nemotron 3 Ultra3.6 s
  12. 36Nemotron 3.5 Lightning6.5 s

62 more models · see the full leaderboard

4 of 78 tested models have no end-to-end response time

All NVIDIA results

Leaderboard
#
1Nemotron 3 Ultra
NVIDIA · served by BaseTen
60.048 / 801360.8 s$0.0057$0.6 / $2.4262k
2Nemotron 3.5 Lightning
NVIDIA · served by Io Net
43.835 / 801160.5 s$0.00049$0.07 / $0.2262k
3Nemotron 3.5 Super VL
NVIDIA · served by Darkbloom
40.032 / 807150.6 s$0.0013$0.15 / $0.51M

NVIDIA: quick answers

What is the best NVIDIA model?
Nemotron 3 Ultra has the highest verification score of the NVIDIA models the leaderboard tested: 60.0% (48 of 80 correct) in the 2026-10-27 run.
What is the fastest NVIDIA model?
Nemotron 3.5 Super VL is the fastest NVIDIA model the leaderboard tested, at a median 715 output tokens per second.
What is the cheapest NVIDIA model?
Nemotron 3.5 Lightning has the lowest billed cost per question of the NVIDIA models the leaderboard tested: $0.00049.
Which NVIDIA models do you test?
Nemotron 3 Ultra, Nemotron 3.5 Lightning, Nemotron 3.5 Super VL.