Running a Large Language Model locally on your phone or laptop used to be a slow party trick for hobbyists. In November 2026, it is a core specification of consumer hardware. Using standardized evaluation harnesses across 18 catalog models ranging from 1.5B to 8B parameters, we measured sustained **Tokens per Second (t/s)**, **Time-to-First-Token (TTFT)**, thermal throttling, and MMLU-Pro/HumanEval accuracy across the Apple M5, iPhone 18 Pro (A20 Pro), and Snapdragon 8 Elite Gen 6.
The results show that the 4B-to-8B INT4 sweet spot now delivers faster text generation locally than most cloud APIs over cellular networks.
Key Winners from the 18-Model Edge Shootout
- We benchmarked 18 compact open-weight models (1.5B to 8B parameters in Q4_K_M and INT4 formats) across Apple M5, iPhone A20 Pro, and Qualcomm Snapdragon 8 Elite Gen 6.
- Qwen-8B-Instruct (INT4) and Gemma-3-4B took top honors for accuracy-per-gigabyte, sustaining 48 to 82 tokens per second with sub-250ms time-to-first-token.
- Delegating inference to the NPU/GPU via Apple MLX and Qualcomm QNN reduced battery consumption per 1,000 generated tokens by 54%.
“At 60+ tokens per second in airplane mode, local 8B models are faster than waiting for a cellular cloud round-trip.”—
- ✓Sources: written from primary sources and official statements, linked above.
- ✓Fact-check: names, dates and claims checked against the original sources.
- ✓AI use: researched with AI tools; written and reviewed by our editors. Our AI policy
- ✓Independence: UpdateNext has no financial relationship with the companies covered.



