ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Ling 3.1 Flash vs Nemotron 3.5 Lightning: accuracy, speed and price

Similar accuracy; Nemotron 3.5 Lightning costs less per question.

inclusionAILing 3.1 Flash41.2% score · #54$0.00061 per question7 tok/sNVIDIANemotron 3.5 Lightning43.8% score · #51$0.00049 per question116 tok/s

Verdict

Built from the measurements below

  • Accuracy is too close to call: Ling 3.1 Flash scored 41.2% and Nemotron 3.5 Lightning 43.8% on the same 80 questions, a gap within what chance alone could produce.
  • Nemotron 3.5 Lightning is 1.3× cheaper per question as billed ($0.00049 vs $0.00061), reasoning tokens included.
  • Nemotron 3.5 Lightning generates text 16× faster: 116 tokens per second against 7.
  • Nemotron 3.5 Lightning starts answering sooner: 0.5 s to the first token, against 1.1 s for Ling 3.1 Flash.

Which should you choose?

Choose Nemotron 3.5 Lightning for the lower cost per correct answer ($0.0011 against $0.0015), faster streaming, a quicker first token, open weights you can run yourself and more hosts to choose from on OpenRouter (4 against 2). The widest subject gap is in engineering: Nemotron 3.5 Lightning answered 4 of 15 correctly, Ling 3.1 Flash 1 of 15. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap -2.5 points (z = -0.32). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureLing 3.1 FlashNemotron 3.5 LightningEdge
Verification scoreHigher is better41.2%43.8%Too close to call
Cost per questionLower is better$0.00061$0.00049Nemotron 3.5 Lightning
Output speedHigher is better7116Nemotron 3.5 Lightning
Time to first tokenLower is better1.1 s0.5 sNemotron 3.5 Lightning
End-to-end response timeLower is better81.3 s6.5 sNemotron 3.5 Lightning
Tokens per questionLower is better2,6162,846Similar
Context windowHigher is better262k262kSimilar

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Ling 3.1 Flash first, Nemotron 3.5 Lightning second

Verification scorehigher is better
Ling 3.1 Flash41.2%
Nemotron 3.5 Lightning43.8%
Cost per questionlower is better
Ling 3.1 Flash$0.00061
Nemotron 3.5 Lightning$0.00049
Output speedhigher is better
Ling 3.1 Flash7
Nemotron 3.5 Lightning116
Time to first tokenlower is better
Ling 3.1 Flash1.1 s
Nemotron 3.5 Lightning0.5 s
End-to-end response timelower is better
Ling 3.1 Flash81.3 s
Nemotron 3.5 Lightning6.5 s
Tokens per questionlower is better
Ling 3.1 Flash2,616
Nemotron 3.5 Lightning2,846
Context windowhigher is better
Ling 3.1 Flash262k
Nemotron 3.5 Lightning262k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Mercury 2.5Phi 4Nemotron 3.5 LightningGranite 4.2 8BHy3GLM 5.3 FlashTrinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120b

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationLing 3.1 FlashNemotron 3.5 Lightning
CreatorinclusionAINVIDIA
Released2 Oct 202611 Aug 2026
Weights—Open weights
ReasoningYesYes
Inputtexttext
Outputtexttext
Context window262k tokens262k tokens
Knowledge cutoff——
List price in / out per 1M$0.075 / $0.225$0.07 / $0.2
Blended price per 1M (3:1)$0.11$0.10
Cheapest hostDeepInfra · $0.075 / $0.225Darkbloom · $0.025 / $0.12
Hosts on OpenRouter24
Last tested27 Oct 202626 Oct 2026

Compare any two models →

Ling 3.1 Flash vs Nemotron 3.5 Lightning: frequently asked questions

Is Ling 3.1 Flash better than Nemotron 3.5 Lightning?
On accuracy they are statistically tied: Ling 3.1 Flash scored 41.2% and Nemotron 3.5 Lightning 43.8% on the same 80 questions from open datasets. Similar accuracy; Nemotron 3.5 Lightning costs less per question.
Which is cheaper, Ling 3.1 Flash or Nemotron 3.5 Lightning?
Nemotron 3.5 Lightning is cheaper: $0.00049 per question as billed, against $0.00061. List prices: Ling 3.1 Flash $0.075 in / $0.225 out, Nemotron 3.5 Lightning $0.07 in / $0.2 out per million tokens.
Which is faster, Ling 3.1 Flash or Nemotron 3.5 Lightning?
Nemotron 3.5 Lightning is faster at 116 output tokens per second, against 7. Time to first token: 1.1 s for Ling 3.1 Flash, 0.5 s for Nemotron 3.5 Lightning.
Which has the larger context window, Ling 3.1 Flash or Nemotron 3.5 Lightning?
Both accept about the same amount of context: 262k and 262k tokens.