Effective Protobuf: Everything You Wanted To Know, But Never Dared To Ask
summary
A production debugger serializes deep runtime snapshots (customers ask for up to 20 levels deep) into protobufs, faces >100 megabyte messages, and optimizes by deduplicating strings into IDs, doing minimal synchronous copies then background serialization, merging and reusing fields with bitwise packing, and favoring small field ids (recommended field ids 1–15). Builds use protoc/proto3, consider the native C extension, and report measured wins of about 40% latency and 50% size reductions.
AI-written summary · about
chapters (24)
- 0:00Introduction
- 2:37Variables
- 3:08Memory
- 5:18Considerations
- 8:04Popular frameworks
- 9:31Why use Protobuf
- 11:16When not to use Protobuf
- 12:18Protobuf overview
- 14:36Primitives
- 15:51Why Protobuf
- 16:47Creating a Protobuf
- 17:44Setting a breakpoint
- 17:54Local variables
- 19:01Optimization goals
- 19:36Serialization
- 20:05Strings
- 21:01Serializing
- 21:31Key Value
- 21:54Key Variants
- 22:20Field IDs
- 22:33Reducing fields
- 23:14Merging fields
- 24:59Oneoff fields
- 26:02Conclusion
related talks
30mPyrobuf and the Tyranny of Speed
30mThe Messaging of Things
50mPerformance tricks learned from low level developers
29mPrototyping Python Microservices in Production
17mHigh Performance Python Microservice Communication
28m"Performant Python"
54mExtending MicroPython with Rust: Two Worlds Collide
23mReducing your memory footprint by 75% with 6 lines of code