Invalid byte sequence in UTF-8
summary
Explains why UTF‑8 is a 1–4 byte variable‑length encoding and how Ruby stores bytes plus an encoding label, showing concrete APIs: force_encoding (changes label only), encode (transcodes), scrub (replaces invalid bytes). Demonstrates three real bugs: Socket.gethostname returning ASCII‑8BIT bytes fixed by forcing UTF‑8, a MySQL migration failing on a truncated UTF‑16 high surrogate that required scrubbing, and an email image URL break caused by Addressable URI changing Unicode normalization forms.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker welcome
- 3:53Computers and bytes basics
- 8:48Limitations of ASCII and code pages
- 13:23Unicode and UTF-8 explained
- 21:48Ruby string encoding handling
- 23:11First bug: Socket hostname encoding error
- 26:42UTF-16 and UTF-32 encoding overview
- 29:54Second bug: High surrogate in UTF-8
- 32:03Normalization forms in Unicode
- 33:52Third bug: Invalid signature in URLs
- 35:43Key takeaways and resources
related talks
32mHow to Become an Encoding Champion
34mThe Three-Encoding Problem
01h 10mInteresting Characters
54mSTRINGS ATTACHED: FROM ASCII TO 🙊🙈🙉
35mRuby and Unicode, what could go wr�ng?
46mASCII to Unicode, what can possibly go w̵̳̓r̸̡̽ǒ̴̝n̴̬ͣg̶ͤ͜?
56mUnicode - going down the rabbit hole
05mLet's start at the beginning - bits to character encoding in R