What’s In A Bit – Designing, Using And Reverse-engineering Binary File Formats
summary
Covers how individual bits scale to 128-bit identifiers, practical encodings (variable-length integers, run-length and back-reference compression), and tradeoffs between memory-mapped files and streaming or loadable formats. Shows design steps: choose working set, add magic/version, pick indexes, iterate with target hardware. Demonstrates reverse engineering a 1990s game: writes an FLX archive extractor, finds a 0xC000 (3072)‑entry index, decodes 9‑ and 6‑byte tables and shape records, reconstructs images from run-length + back-reference compression and successfully renders game levels.
AI-written summary · about
chapters (22)
- 0:00Start
- 0:12Basics
- 2:50C64 Palette
- 3:24Web Safe Palette
- 3:4416 Bits
- 6:01How Much Would It Actually Take To Store Data
- 13:44Run Link Encoding
- 20:03Protobuf
- 22:11Why Are You Doing Binary
- 23:34General Types of File Formats
- 24:11Header with Flat Data
- 27:51Record Sequence
- 31:09Memory Mapped Files
- 33:21Loadable File
- 34:52Designing a Binary File Format
- 35:19Working Set
- 35:31What a Working Set Is
- 37:33Difference in Video File Formats
- 40:15Start with a Magic Value and a Version Number
- 42:12Iterate Your Design
- 50:29Identify File Structures
- 58:00Run Length Encoding
related talks
01h 33mBinary File Format Engineering and Reverse Engineering
37mBehind Enemy Lines - Reverse Engineering C++ in Modern Ages
01h 27mFile IO - Past, Present and Future
24m(Ab)using Compiler Tools
01h 24mOptimising a small real-world C++ application
55mUsing Integers as Arrays of Bitfields a.k.a. SWAR Techniques
01h 04mBeyond the Floating Point Computations - What’s New, What’s Useful?
01h 08mFrom Numerical Cosmology to Efficient Bit Abstractions for the Standard Library