Python's Transition to Multilingual Programming
summary
Argues that Python 3 really does make all strings be Unicode and demonstrates with a pandas demo with Japanese names showing Python 2’s 8‑bit failures, normalization and lookup surprises, and why bilingual codecs break on networks. Shows concrete changes: CPython uses a UTF-32 fixed-width storage internally to reuse array algorithms and restores binary interpolation returned for bytes. Notes user‑space now mostly works but system boundaries still fail sometimes: macOS is solid, Windows mixed, and POSIX locales can misreport encodings.
AI-written summary · about
chapters (16)
- 0:00The Rise of Multilingual Programming
- 0:31Make all strings be Unicode (PEP 3000, 2004)
- 1:44Why change the default?
- 8:49One small part of a much larger pattern
- 9:30The long shadow of the telegraph
- 14:17Extended ASCII
- 15:16A cornucopia of codecs
- 15:58Bilingual computing
- 17:27Multilingual computing
- 20:08ASCII compatibility (UTF-8)
- 21:40Basic multilingual plane (UTF-16-LE)
- 22:41O(1) random code point access (UTF-32)
- 24:29Where does Python 3 stand?
- 28:04System boundaries: Mac OS X
- 29:26System boundaries: Windows
- 30:35System boundaries: Linux & other POSIX based platform
related talks
35mPython, Locales and Writing Systems
35mText Is Not Text: The History of Strings in Python
28mMaking Acquaintances with Exotic Encodings
26mPython 2 to 3: How to Upgrade and What Features to Start Using
32mHow (not) to upgrade a platform
44mA 🐍's guide to Unicode
01h 00mKeynote: Python now and in the future
01h 12mPython 3: the Next Generation (is here already)