ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Head to head · tested 27 Oct 2026

Gemini 3.5 Flash Lite vs Gemma 4 31B: accuracy, speed and price

Gemini 3.5 Flash Lite is more accurate; Gemma 4 31B is cheaper.

GoogleGemini 3.5 Flash Lite75.0% score · #15$0.0034 per question409 tok/sGoogleGemma 4 31B30.0% score · #64$0.0014 per question58 tok/s

Verdict

Built from the measurements below

  • Gemini 3.5 Flash Lite is more accurate: 75.0% against 30.0% on the same 80 questions, a 45.0-point gap that is statistically significant.
  • Gemma 4 31B is 2.4× cheaper per question as billed ($0.0014 vs $0.0034), reasoning tokens included.
  • Gemini 3.5 Flash Lite generates text 7.0× faster: 409 tokens per second against 58.
  • Gemma 4 31B starts answering sooner: 0.6 s to the first token, against 1.1 s for Gemini 3.5 Flash Lite.
  • Gemini 3.5 Flash Lite accepts more context: 1M tokens against 262k.

Which should you choose?

Choose Gemini 3.5 Flash Lite for the most accurate answers of the two, faster streaming, shorter outputs (about 1,311 tokens per question against 3,307) and a 1M-token context window. Choose Gemma 4 31B for a quicker first token, open weights you can run yourself and more hosts to choose from on OpenRouter (13 against 2). The widest subject gap is in engineering: Gemini 3.5 Flash Lite answered 10 of 15 correctly, Gemma 4 31B 0 of 15. With so few questions per subject, treat this as a lead to test on your own prompts, not a result.

Accuracy gap +45.0 points (z = 5.70). A winner is called only when z is at least 1.96, the usual 95% threshold. Speed, latency and price need a 10% margin.

Head to head

Same questions, same prompt, same API

MeasureGemini 3.5 Flash LiteGemma 4 31BEdge
Verification scoreHigher is better75.0%30.0%Gemini 3.5 Flash Lite
Cost per questionLower is better$0.0034$0.0014Gemma 4 31B
Output speedHigher is better40958Gemini 3.5 Flash Lite
Time to first tokenLower is better1.1 s0.6 sGemma 4 31B
End-to-end response timeLower is better2.8 s12.6 sGemini 3.5 Flash Lite
Tokens per questionLower is better1,3113,307Gemini 3.5 Flash Lite
Context windowHigher is better1M262kGemini 3.5 Flash Lite

Side by side

Bars are scaled to the larger of the two values. The winner on each measure is marked in bold.

Every measure

Gemini 3.5 Flash Lite first, Gemma 4 31B second

Verification scorehigher is better
Gemini 3.5 Flash Lite75.0%
Gemma 4 31B30.0%
Cost per questionlower is better
Gemini 3.5 Flash Lite$0.0034
Gemma 4 31B$0.0014
Output speedhigher is better
Gemini 3.5 Flash Lite409
Gemma 4 31B58
Time to first tokenlower is better
Gemini 3.5 Flash Lite1.1 s
Gemma 4 31B0.6 s
End-to-end response timelower is better
Gemini 3.5 Flash Lite2.8 s
Gemma 4 31B12.6 s
Tokens per questionlower is better
Gemini 3.5 Flash Lite1,311
Gemma 4 31B3,307
Context windowhigher is better
Gemini 3.5 Flash Lite1M
Gemma 4 31B262k

Score vs. cost per question

Both models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-5.4GPT-6 Lunagpt-oss-120bLlama 4 Maverick

Specifications and pricing

From OpenRouter's catalogue, plus the leaderboard's test dates

SpecificationGemini 3.5 Flash LiteGemma 4 31B
CreatorGoogleGoogle
Released21 Jul 20262 Apr 2026
Weights—Open weights
ReasoningYesYes
Inputtext, image, video, file, audioimage, text, video
Outputtexttext
Context window1M tokens262k tokens
Knowledge cutoff——
List price in / out per 1M$0.3 / $2.5$0.2 / $0.4
Blended price per 1M (3:1)$0.85$0.25
Cheapest hostGoogle · $0.15 / $1.25DeepInfra · $0.16 / $0.32
Hosts on OpenRouter213
Last tested26 Oct 202627 Oct 2026

Compare any two models →

Gemini 3.5 Flash Lite vs Gemma 4 31B: frequently asked questions

Is Gemini 3.5 Flash Lite better than Gemma 4 31B?
Gemini 3.5 Flash Lite is more accurate: it answered 60 of 80 questions correctly (75.0%) against 24 of 80 (30.0%) for Gemma 4 31B. Gemini 3.5 Flash Lite is more accurate; Gemma 4 31B is cheaper.
Which is cheaper, Gemini 3.5 Flash Lite or Gemma 4 31B?
Gemma 4 31B is cheaper: $0.0014 per question as billed, against $0.0034. List prices: Gemini 3.5 Flash Lite $0.3 in / $2.5 out, Gemma 4 31B $0.2 in / $0.4 out per million tokens.
Which is faster, Gemini 3.5 Flash Lite or Gemma 4 31B?
Gemini 3.5 Flash Lite is faster at 409 output tokens per second, against 58. Time to first token: 1.1 s for Gemini 3.5 Flash Lite, 0.6 s for Gemma 4 31B.
Which has the larger context window, Gemini 3.5 Flash Lite or Gemma 4 31B?
Gemini 3.5 Flash Lite, with 1M tokens against 262k.