ResearchLLM Verifications Leaderboard

Data as of · 78 models, 80 questions · reviewed by Suhas Bhairav · methodology

Provider

Qwen AI models

9 models tested, compared with 69 models from other labs.

Best verification score75.0Qwen3.8 Max (0902) · #15 of 78Best output speed240Qwen3.5 397B A17B · #5 of 78Best time to first token0.4 sQwen3.6 35B A3B · #6 of 78Best cost per question$0.0017Qwen3 Coder Next · #33 of 78
Qwen#15Qwen3.8 Max (0902)75.0% verification score46 tok/s1.2 s first token$0.011 / questionQwen#34Qwen3.8 27B63.8% verification score49 tok/s0.5 s first token$0.0043 / questionQwen#36Qwen3 Coder Next62.5% verification score106 tok/s0.4 s first token$0.0017 / questionQwen#38Qwen3.7 Plus60.0% verification score62 tok/s1.1 s first token$0.0033 / questionQwen#43Qwen3.8 2.4T A95B57.5% verification score44 tok/s3.1 s first token$0.015 / questionQwen#59Qwen3.6 35B A3B37.5% verification score135 tok/s0.4 s first token$0.0037 / questionQwen#62Qwen3.5 397B A17B32.5% verification score240 tok/s0.6 s first token$0.013 / question

Score vs. cost per question

Qwen models in color, every other model in grey

Most attractive quadrant0255075100$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Verification score (%) →Qwen3.8 2.4T A95BCommand AReka Flash 3Gemma 4 26B A4BGemma 4 31BQwen3.5 397B A17BHermes 4 405BGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Phi 4Granite 4.2 8BHy3GLM 5.3 FlashSolar Pro 4Trinity Large ThinkingMiniMax M3Mistral Large 4Claude Opus 5.5GPT-6.1 SolGemini 4 ArgonGPT-6 Lunagpt-oss-120bLlama 4 MaverickQwen3.8 Max (0902)

Output speed vs. cost per question

Qwen models in color, every other model in grey

Most attractive quadrant0200400600800$0.00001$0.0001$0.001$0.01$0.10Cost per question (USD, log) →Output speed (tok/s) →Nemotron 3.5 Super VLQwen3.5 397B A17BMuse WonderGrok 4.7o3gpt-oss-20bMinistral 3 3B 2512Nova Micro 1.0Trinity Large ThinkingInkling SmallGemini 3.5 Flash LiteMuse Spark OpenGPT-6 Luna

Verification score

Qwen models highlighted against every model in this run · higher is better

  1. 1Claude Opus 5.585.0
  2. 2Gemini 4 Argon83.8
  3. 3GPT-5.3-Codex82.5
  4. 3Gemini 3.8 Flash82.5
  5. 5Claude Fable 5.181.2
  6. 6Grok 4.780.0
  7. 7Muse Wonder78.8
  8. 7Claude Sonnet 5.578.8
  9. 9Gemini 3.1 Pro Preview77.5
  10. 9Muse Spark Open77.5
  11. 15Qwen3.8 Max (0902)75.0
  12. 34Qwen3.8 27B63.8
  13. 36Qwen3 Coder Next62.5
  14. 38Qwen3.7 Plus60.0
  15. 43Qwen3.8 2.4T A95B57.5
  16. 59Qwen3.6 35B A3B37.5
  17. 62Qwen3.5 397B A17B32.5

55 more models · see the full leaderboard

6 of 78 tested models have no verification score

Output speed

Qwen models highlighted against every model in this run · higher is better

  1. 1Nemotron 3.5 Super VL715
  2. 2Gemini 3.5 Flash Lite409
  3. 3Trinity Large Thinking329
  4. 4Ministral 3 3B 2512252
  5. 5Qwen3.5 397B A17B240
  6. 6Inkling Small233
  7. 7Muse Spark Open220
  8. 8Claude Haiku 5.5192
  9. 9Nova Micro 1.0180
  10. 10Muse Wonder175
  11. 22Qwen3.6 35B A3B135
  12. 32Qwen3.5-122B-A10B108
  13. 34Qwen3 Coder Next106
  14. 52Qwen3.7 Plus62
  15. 60Qwen3.8 27B49
  16. 61Qwen3.8 Max (0902)46
  17. 62Qwen3.8 2.4T A95B44

55 more models · see the full leaderboard

6 of 78 tested models have no output speed

Time to first token

Qwen models highlighted against every model in this run · lower is better

  1. 1Ministral 3 3B 25120.3 s
  2. 2Ministral 3 8B 25120.3 s
  3. 3Hermes 4 405B0.3 s
  4. 4Kimi K2.7 Code0.4 s
  5. 5MiMo-V2.6-Flash0.4 s
  6. 6Qwen3.6 35B A3B0.4 s
  7. 7Qwen3 Coder Next0.4 s
  8. 8Mistral Medium 3.50.5 s
  9. 9Phi 40.5 s
  10. 10Nova Micro 1.00.5 s
  11. 19Qwen3.8 27B0.5 s
  12. 27Qwen3.5 397B A17B0.6 s
  13. 48Qwen3.7 Plus1.1 s
  14. 51Qwen3.8 Max (0902)1.2 s
  15. 54Qwen3.5-122B-A10B1.2 s
  16. 68Qwen3.8 2.4T A95B3.1 s

58 more models · see the full leaderboard

4 of 78 tested models have no time to first token

Cost per question

Qwen models highlighted against every model in this run · lower is better

  1. 1Granite 4.0 Micro$0.000051
  2. 2gpt-oss-20b$0.000066
  3. 3Granite 4.2 8B$0.000077
  4. 4Nova Micro 1.0$0.000087
  5. 5Mercury 2.5$0.00011
  6. 6Phi 4$0.00012
  7. 7gpt-oss-120b$0.00013
  8. 8GLM 5.3 Flash$0.00015
  9. 9Ministral 3 3B 2512$0.00019
  10. 10GPT-6 Luna$0.00025
  11. 33Qwen3 Coder Next$0.0017
  12. 42Qwen3.7 Plus$0.0033
  13. 46Qwen3.6 35B A3B$0.0037
  14. 47Qwen3.8 27B$0.0043
  15. 61Qwen3.8 Max (0902)$0.011
  16. 65Qwen3.5 397B A17B$0.013
  17. 67Qwen3.8 2.4T A95B$0.015

55 more models · see the full leaderboard

6 of 78 tested models have no cost per question
  • LFM2.5-2.6B: no verification score (80 of 80 questions failed on the host's side)
  • Ling 3.0 Flash VL: no verification score (not yet run on the current question suite)
  • Qwen3.5-122B-A10B: no verification score (not yet run on the current question suite)
  • Qwen3 Next 80B A3B Thinking: no verification score (not yet run on the current question suite)
  • Nova Premier 1.0: no verification score (not yet run on the current question suite)
  • GPT-5.5 Pro: no verification score (not yet run on the current question suite)

End-to-end response time

Qwen models highlighted against every model in this run · lower is better

  1. 1Nemotron 3.5 Super VL1.5 s
  2. 2Ministral 3 3B 25121.6 s
  3. 3Nova Micro 1.02.1 s
  4. 4Trinity Large Thinking2.7 s
  5. 5Gemini 3.5 Flash Lite2.8 s
  6. 6DeepSeek V4 Flash Vision Exp3.2 s
  7. 7Muse Spark Open3.2 s
  8. 8GLM 5.33.4 s
  9. 9Ministral 3 8B 25123.5 s
  10. 10Qwen3.5 397B A17B3.5 s
  11. 17Qwen3 Coder Next4.2 s
  12. 31Qwen3.6 35B A3B5.6 s
  13. 58Qwen3.8 27B10.6 s
  14. 61Qwen3.8 Max (0902)12.2 s
  15. 62Qwen3.7 Plus12.4 s
  16. 65Qwen3.8 2.4T A95B14.8 s
  17. 67Qwen3.5-122B-A10B18.0 s

57 more models · see the full leaderboard

4 of 78 tested models have no end-to-end response time

All Qwen results

Leaderboard
#
1Qwen3.8 Max (0902)
Qwen · served by Alibaba
75.060 / 80461.2 s$0.011$2 / $61M
2Qwen3.8 27B
Qwen · served by Modal
63.851 / 80490.5 s$0.0043$0.2 / $2.51M
3Qwen3 Coder Next
Qwen · served by Parasail
62.550 / 801060.4 s$0.0017$0.12 / $0.8262k
4Qwen3.7 Plus
Qwen · served by Alibaba
60.048 / 80621.1 s$0.0033$0.32 / $1.281M
5Qwen3.8 2.4T A95B
Qwen · served by Venice
57.546 / 80443.1 s$0.015$2 / $61M
6Qwen3.6 35B A3B
Qwen · served by DeepInfra
37.530 / 801350.4 s$0.0037$0.1 / $0.95262k
7Qwen3.5 397B A17B
Qwen · served by Phala
32.526 / 802400.6 s$0.013$0.5 / $3.6262k
8Qwen3.5-122B-A10B
Qwen · served by SiliconFlow
—0 / 01081.2 s—$0.4 / $3.2262k
9Qwen3 Next 80B A3B Thinking—0 / 0———$0.15 / $1.2262k

Qwen: quick answers

What is the best Qwen model?
Qwen3.8 Max (0902) has the highest verification score of the Qwen models the leaderboard tested: 75.0% (60 of 80 correct) in the 2026-10-27 run.
What is the fastest Qwen model?
Qwen3.5 397B A17B is the fastest Qwen model the leaderboard tested, at a median 240 output tokens per second.
What is the cheapest Qwen model?
Qwen3 Coder Next has the lowest billed cost per question of the Qwen models the leaderboard tested: $0.0017.
Which Qwen models do you test?
Qwen3.8 Max (0902), Qwen3.8 27B, Qwen3 Coder Next, Qwen3.7 Plus, Qwen3.8 2.4T A95B, Qwen3.6 35B A3B, Qwen3.5 397B A17B, Qwen3.5-122B-A10B, Qwen3 Next 80B A3B Thinking.