ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

gpt-oss-20b vs Nemotron 3.5 Lightning: accuracy, speed and price

Similar accuracy; gpt-oss-20b costs less per question.

OpenAIgpt-oss-20b51.2% score · #46$0.000066 per question103 tok/sNVIDIANemotron 3.5 Lightning43.8% score · #51$0.00049 per question116 tok/s

Verdict

Built from the measurements below

  • Accuracy is too close to call: gpt-oss-20b scored 51.2% and Nemotron 3.5 Lightning 43.8% on the same 80 questions, a gap within what chance alone could produce.
  • gpt-oss-20b is 7.4× cheaper per question as billed ($0.000066 vs $0.00049), reasoning tokens included.
  • Nemotron 3.5 Lightning generates text 1.1× faster: 116 tokens per second against 103.
  • Nemotron 3.5 Lightning starts answering sooner: 0.5 s to the first token, against 0.7 s for gpt-oss-20b.
  • Nemotron 3.5 Lightning accepts more context: 262k tokens against 131k.

Which should you choose?

Choose gpt-oss-20b for the lower cost per correct answer ($0.00013 against $0.0011), shorter outputs (about 472 tokens per question against 2,846) and more hosts to choose from on OpenRouter (10 against 4). Choose Nemotron 3.5 Lightning for faster streaming, a quicker first token and a 262k-token context window.

Accuracy gap +7.5 points (z = 0.95). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

Measuregpt-oss-20bNemotron 3.5 LightningEdge
Verification scoreHigher is better51.2%43.8%Too close to call
Cost per questionLower is better$0.000066$0.00049gpt-oss-20b
Output speedHigher is better103116Nemotron 3.5 Lightning
Time to first tokenLower is better0.7 s0.5 sNemotron 3.5 Lightning
End-to-end response timeLower is better4.0 s6.5 sgpt-oss-20b
Tokens per questionLower is better4722,846gpt-oss-20b
Context windowHigher is better131k262kNemotron 3.5 Lightning

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

gpt-oss-20b first, Nemotron 3.5 Lightning second

Verification scorehigher is better
gpt-oss-20b51.2%
Nemotron 3.5 Lightning43.8%
Cost per questionlower is better
gpt-oss-20b$0.000066
Nemotron 3.5 Lightning$0.00049
Output speedhigher is better
gpt-oss-20b103
Nemotron 3.5 Lightning116
Time to first tokenlower is better
gpt-oss-20b0.7 s
Nemotron 3.5 Lightning0.5 s
End-to-end response timelower is better
gpt-oss-20b4.0 s
Nemotron 3.5 Lightning6.5 s
Tokens per questionlower is better
gpt-oss-20b472
Nemotron 3.5 Lightning2,846
Context windowhigher is better
gpt-oss-20b131k
Nemotron 3.5 Lightning262k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

Specificationgpt-oss-20bNemotron 3.5 Lightning
CreatorOpenAINVIDIA
Released5 Aug 202511 Aug 2026
WeightsOpen weightsOpen weights
ReasoningYesYes
Inputtexttext
Outputtexttext
Context window131k tokens262k tokens
Knowledge cutoff30 Jun 2024—
List price in / out per 1M$0.018 / $0.09$0.07 / $0.2
Blended price per 1M (3:1)$0.04$0.10
Cheapest hostDarkbloom · $0.018 / $0.09Darkbloom · $0.025 / $0.12
Hosts on OpenRouter104
Last tested26 Oct 202626 Oct 2026

Compare any two models →

gpt-oss-20b vs Nemotron 3.5 Lightning: frequently asked questions

Is gpt-oss-20b better than Nemotron 3.5 Lightning?
On accuracy they are statistically tied: gpt-oss-20b scored 51.2% and Nemotron 3.5 Lightning 43.8% on the same 80 questions from open datasets. Similar accuracy; gpt-oss-20b costs less per question.
Which is cheaper, gpt-oss-20b or Nemotron 3.5 Lightning?
gpt-oss-20b is cheaper: $0.000066 per question as billed, against $0.00049. List prices: gpt-oss-20b $0.018 in / $0.09 out, Nemotron 3.5 Lightning $0.07 in / $0.2 out per million tokens.
Which is faster, gpt-oss-20b or Nemotron 3.5 Lightning?
Nemotron 3.5 Lightning is faster at 116 output tokens per second, against 103. Time to first token: 0.7 s for gpt-oss-20b, 0.5 s for Nemotron 3.5 Lightning.
Which has the larger context window, gpt-oss-20b or Nemotron 3.5 Lightning?
Nemotron 3.5 Lightning, with 262k tokens against 131k.