Gemma on mobile and web. Best and worst practices
summary
Mobile and web deployment focuses on concrete tradeoffs and tools not fluffy promises: use Gemma 3 1B quantized to 529 megabytes (zipped under 400 MB) to run on devices with ~4 GB RAM, get ~2,500 tokens per second prefill and ~56 tokens/sec decode, bundle models for privacy, latency and zero cloud cost, or use system models for higher quality but less customization, use RAG with the Google AI Edge RAG SDK and MediaPipe / LiteRT for on-device retrieval and inference.
AI-written summary · about
chapters (8)
- 0:00Run AI on device
- 2:07System vs in-app models
- 3:51Gemma 3 1B benefits
- 4:54Language-driven interactions
- 6:40Text generation use cases
- 9:55Best practices for Gemma 1B
- 11:21On-device RAG with Edge SDK
- 12:04Integrating Gemma
related talks
08mDemo: Gemma on-device with MediaPipe and TensorFlow Lite
40mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model
44mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model
39mFlutter Project with Gemma: A State-of-the-Art Open Model
10mDemo: Using Gemma with the Hugging Face ecosystem
19mGemma 4 Deep Dive
01h 21mTLMs: Tiny LLMs and Agents on Edge Devices with LiteRT-LM
15mGemma, DeepMind's Family of Open Models