The Three-Encoding Problem
summary
Claims Ruby ships more than 100 encodings and most apps actually touch three encodings: UTF-8 for literals, US-ASCII for converted numbers/symbols, and ASCII-8BIT/binary for IO. Demonstrations use Encoding.list, String#encoding, String#valid_encoding, String#encode with error handlers and the dangerous force_encoding hack. Measured surprises include how concatenation and compatibility checks can scan whole strings. Recent work in Ruby 3.2 adjusts the code range metadata and string concatenation logic to reduce those scans and measurably improve performance.
AI-written summary · about
chapters (7)
- 0:00Start
- 5:14American Standard Code for Information Interchange
- 9:05Unicode Transformation Formats
- 13:17Transcoding Error: Undefined Conversion
- 14:38Handling Transcoding Errors
- 16:14Broken Strings
- 30:13Behavioral Impact
related talks
38mInvalid byte sequence in UTF-8
32mHow to Become an Encoding Champion
35mRuby and Unicode, what could go wr�ng?
39mState of the Ruby
38mTen Unicode Characters You Should Know About as a Programmer
40mWeird Ruby
24mThe Power and Responsibility of Unicode Adoption
32mWhy We Can't Have Nice Things: Unicode, Dates, and Names