Building a Handwriting Recognition System for the New York Times Crossword
summary
Builds an on-device handwriting recognizer for the New York Times mini crossword using a per-square tiny canvas, touchpoint tracking and a mutex timing lock that converts strokes to bitmaps for inference. Moves from edge detectors and Google ML Kit to a TensorFlow Lite convolutional model ~100KB, trains on EMNIST with synthetic augmentation from 60k up to 1.2–2 million augmented samples, runs client-side inference and QA, reaches about 91% validation accuracy on the dataset while letters remain harder than digits.
AI-written summary · about
chapters (9)
- 0:00Handwriting recognition project
- 2:27Handwriting input setup
- 5:59Handwriting system architecture
- 7:24Classical methods to ML
- 11:18TensorFlow Lite model
- 16:16Synthetic data challenges
- 19:46Model complexity and validation
- 22:37Next steps and improvements
- 23:42Comparison with ML Kit
related talks
44mBuilding Handwriting Recognition System for the NY Times Crossword
40mBuilding a Handwriting Recognition System
44mHello Deep Learning
19mReacTex: using React Native and Neural Networks to recognize handwritten equations
06mWrite, Attend and Spell: Streaming End-to-end Free-style Handwriting Recognition Using Smartwatches
01h 07mDeep Learning with TensorFlow
51mDeciphering doctor's handwriting with deep learning
24mSIGN TO TEXT Unleashing the Power of Google AI for real time ASL transcription