Hack Weekend: ML models on mobile
summary
Attempts to run GPT-2 inside an iMessage extension over a weekend produce a working prototype only by falling back to server inference because Core ML refuses cyclic graphs: the model uses a while loop control flow and Core ML requires static graphs. The full model has about 117M parameters and ~500 MB on disk but 8‑bit quantization shrinks it to roughly 40 MB; Hugging Face’s Swift/Core ML port works but requires reimplementing tokenization and many low‑level tensor ops in Swift. Swift for TensorFlow looks like the real long‑term solution.
AI-written summary · about
chapters (13)
- 0:00Intro
- 5:06Hack Weekend plan
- 6:58GBT to
- 8:26iMessage
- 9:04Python
- 9:33Complexity
- 11:46The Plan
- 13:01Core ML Hello World
- 14:48Core ML Model
- 17:14Core ML Limitations
- 23:33ML Engineering
- 28:11Neural Engines
- 29:30Motivation
related talks
01h 00mInside GPT – Large Language Models Demystified
39mFrom TensorFlow to ML Kit: power your mobile application with machine learning
31mPower-Up Your App With Apple’s Machine Learning Frameworks
29mFrom TensorFlow to ML Kit: power your Android application with ML
41mInside GPT – The Maths Behind the Magic
49mMobiconf 2018: Vladimir Alekseichenko, DataWorkshop
25mGetting Started with Machine Learning in your iOS apps
29mPushing Machine Learning into your app When TensorFlow and MLKit rule the world