Accepting complex inputs with simple code
summary
Shows how to accept messy byte streams by splitting tokenization from validation, using bufio.Scanner with a custom split to stream tokens, and small state functions that return the next state for parsing. Demonstrates a streaming SSE parser that bubbles scanner and parser errors, uses io.LimitReader for bounds, and can emit channels for parallel processing. Uses differential fuzzing to find mismatches — one audit finds 36 bugs in fastHTTP compared to the stdlib.
AI-written summary · about
chapters (14)
- 0:00Introduction to input parsing
- 1:09Context-sensitive grammar challenge
- 2:34Problems with regular expressions
- 3:40Flaws in initial parsing attempt
- 5:27Handling streaming data
- 7:54State machine approach
- 10:22Benefits of state functions
- 12:29Fuzzing for robust validation
- 15:41Real-world example: Server-Sent Events
- 19:32Layered parsing architecture
- 20:37Parser generators overview
- 21:25Q&A: Regex use cases
- 23:13Q&A: Input processing limits
- 24:38Q&A: Fuzzing strategies
