API Design as a Performance Primitive

on YouTube ↗01h 04m

" summary

Demonstrates a handbuilt streaming JSON parser that replaces encoding/json’s per-token allocation model with a sliding-window, provider-owned buffer window API, cutting token allocations from about 52 to 3 and showing ~10× tokenization throughput versus encoding/json, ~2–3× for a full decoder, and up to 10× with the bytes-oriented API. Uses a custom byteReader/scan/next flow, small reusable buffers (tunable from ~60 bytes to 4–8 KB), eliminates per-byte error checks, and delivers multi-hundred MB/s to low GB/s parsing with near-zero allocations.

AI-written summary · about

" chapters (30)

1. 0:00Introduction by host and speaker welcome
2. 0:47Thesis on API design as performance primitive
3. 1:53Goals and constraints for JSON decoder API design
4. 3:44Why streaming JSON decoding is critical
5. 5:08Complexity of JSON tokenization and parsing
6. 8:17Cost of allocations in hot paths and garbage collection
7. 9:27Understanding JSON tokenization basics
8. 11:00Problems with encoding JSON token API
9. 13:28Benchmarking encoding JSON token overhead
10. 14:51Designing a new token API with byte slices
11. 16:20Introducing package JSON and its API
12. 18:30Benchmarking package JSON tokenization
13. 20:15Challenges of efficient JSON reading
14. 24:08Introducing byte reader for streaming data
15. 27:01Byte reader API and mechanics
16. 30:57Whitespace scanning with byte reader
17. 33:25Benchmarking whitespace scanning performance
18. 35:33Building a JSON scanner on byte reader
19. 36:59Token function and whitespace skipping
20. 39:02Error handling in scanner and byte reader
21. 41:30Benchmarking scanner performance
22. 43:20Performance impact of inlining scanner functions
23. 46:18Validating JSON structure with state machine
24. 47:46Optimizing state transitions with method expressions
25. 50:15Outlining and inlining for decoder performance
26. 52:47Final benchmark results and JSON decoder performance
27. 54:57Key takeaways on API design and performance
28. 57:10Limitations of I/O reader and writer APIs
29. 58:34Introducing byte reader package and future ideas
30. 1:00:19Q&A on error checking and API design trade-offs

" related talks

• 30mCaching, Payloads, and Other Dark ArtsFaris Aziz · JSHeroes 2026
• 30mProcessing large JSON files without running out of memoryItamar Turner Trauring · PyCon US 2025
• 01h 03mExtreme Performance ArchitectureOren Eini · Build Stuff 2017
• 27mHigh Performance ArchitectureMarcel Weiher · App Builders Switzerland 2017
• 41mParsing Safely, from 500MB/s to 2GB/sGeoffroy Couprie · Code Mesh LDN 2018
• 47mToward a High-Performance JSON Protocol: NotesDaniel Austin · JSConf US 2011
• 40mPerf StoriesDavid Mark Clements · Node Week 2017
• 21mHigh Performance APIs through CachingJulien Maingard · Serverless Architecture Conference 2019