Running LLMs on your iPhone: 40 tok/s Gemma 4 with MLX
summary
Demo shows Gemma 4 running locally on iPhone via MLX with concrete results: Gemma 4 8‑bit at ~40 tokens/s streaming offline, integrated through the MLX Swift LM repo, and packaged in the Locally AI app. Recommends 4–8‑bit quantization (under 4‑bit degrades quality), model weights download ~1–3 GB, works on recent and older iPhones (lower throughput), supports tool calling but not full structured generation, and installs via Hugging Face IDs into MLX for production use.
AI-written summary · about
chapters (6)
- 0:00Running LLMs on iPhone
- 1:52What is MLX
- 3:20Downloading models
- 5:46Performance demo of 40 tokens per second
- 7:12Try it yourself
- 9:02Q&A
related talks
35mOn‑Device LLMs on Android Devices 2025
08mDemo: Gemma on-device with MediaPipe and TensorFlow Lite
46mSmall AI is the next Big Thing
35mLocal LLMs on low end hardware - a practical perspective
02h 52mLow Level Technicals of LLMs
16mGemma 4: Compacting Intelligence for the Edge
23m2025: On‑Device AI Goes Mainstream on Android
18mLive demo: Gemma 3 on Jetson Orin Nano