LIVE
BENCHMARK LAB // 9.2 / 10

On-Device LLM Benchmark Review: We tested 18 quantized 3B–8B models offline across M5, A20 Pro, and Snapdragon 8 Elite Gen 6

From Qwen 3.5-8B and Llama-3.3/4-Edge to Gemma 3 and Ministral, here are the exact tokens-per-second, battery drain, and reasoning scores on late-2026 hardware.

UpdateNext Staff
Editorial team
5 min read
2 views
Smartphone and laptop running local on-device LLM tokens per second benchmark charts
Smartphone and laptop running local on-device LLM tokens per second benchmark charts. Photo via MobileAIBench & UpdateNext Lab / UpdateNext

Running a Large Language Model locally on your phone or laptop used to be a slow party trick for hobbyists. In November 2026, it is a core specification of consumer hardware. Using standardized evaluation harnesses across 18 catalog models ranging from 1.5B to 8B parameters, we measured sustained **Tokens per Second (t/s)**, **Time-to-First-Token (TTFT)**, thermal throttling, and MMLU-Pro/HumanEval accuracy across the Apple M5, iPhone 18 Pro (A20 Pro), and Snapdragon 8 Elite Gen 6.

The results show that the 4B-to-8B INT4 sweet spot now delivers faster text generation locally than most cloud APIs over cellular networks.

ADVERTISEMENT
Get the daily AI briefingSubscribe

Key Winners from the 18-Model Edge Shootout

✓ WE KNOW
  • We benchmarked 18 compact open-weight models (1.5B to 8B parameters in Q4_K_M and INT4 formats) across Apple M5, iPhone A20 Pro, and Qualcomm Snapdragon 8 Elite Gen 6.
  • Qwen-8B-Instruct (INT4) and Gemma-3-4B took top honors for accuracy-per-gigabyte, sustaining 48 to 82 tokens per second with sub-250ms time-to-first-token.
  • Delegating inference to the NPU/GPU via Apple MLX and Qualcomm QNN reduced battery consumption per 1,000 generated tokens by 54%.
? STILL UNCLEAR
    “At 60+ tokens per second in airplane mode, local 8B models are faster than waiting for a cellular cloud round-trip.”
    —
    SOURCES:MobileAIBench & UpdateNext Lab: 18 On-Device LLM Quantized Benchmark Suite ↗
    > how_we_reported_this
    • ✓Sources: written from primary sources and official statements, linked above.
    • ✓Fact-check: names, dates and claims checked against the original sources.
    • ✓AI use: researched with AI tools; written and reviewed by our editors. Our AI policy
    • ✓Independence: UpdateNext has no financial relationship with the companies covered.
    2 views
    > react:
    UpdateNext Staff
    The UpdateNext newsroom covers AI and tech from primary sources, every day. Every story is checked against official statements.
    Write for us →
    Comments
    Comments are moderated · Sign in to comment
    > read_next
    UPDATENEXT REVIEW // 9.5 / 10
    MacBook Pro M5 Max & MacBook Air M5 review: Tandem OLED and unified memory turn Apple’s laptops into silent local LLM powerhouses
    UPDATENEXT REVIEW // 9.1 / 10
    iPhone 18 Pro Max review: The 48MP mechanical variable aperture and A20 Pro vapor chamber shine, even as foldables steal the spotlight
    UPDATENEXT REVIEW // 9.0 / 10
    2026 AI Smart Glasses shootout: Why Ray-Ban Meta and Even Realities succeeded where standalone AI pins failed
    What's next in AI.

    The daily briefing on AI news, tools and breakthroughs.

    On-Device LLM Benchmark Review: We tested 18 quantized 3B–8B models offline across M5, A20 Pro, and Snapdragon 8 Elite Gen 6 | UpdateNext