Unicode Best Practices
summary
Recommendations: use use utf-8 for IO everywhere—decode incoming UTF‑8 to character strings, work on characters, then encode UTF‑8 on output; keep source files UTF‑8 and use literal UTF‑8 in HTML; prefer hex numeric entities if escaping. Regex notes: \d matches digits in all scripts so use [0-9] for ASCII digits; use \p{...} properties and script values for precise matching; normalize and case‑fold for comparisons; sort with the Unicode Collation Algorithm. Tests include above 0x7F, 0xFF, 0xFFF (surrogates), emoji, grapheme clusters. Repo: unicode cookbook. case folding
AI-written summary · about
chapters (18)
- 0:00Unicode best practices intro
- 1:15Why follow best practices
- 3:24UTF-8 for source code I/O
- 7:16Readable regex with UTF-8
- 10:09Ligatures in regex handling
- 12:37Unicode regex character classes
- 15:07When to use regex properties
- 20:10Grapheme clusters vs code points
- 24:09Unicode character properties
- 30:55Case folding & normalization
- 34:10Normalize input/output
- 35:32Fix locale sorting bugs
- 36:40Unicode Collation Algorithm
- 38:25Test Unicode awareness
- 40:12Test high code points
- 41:16UTF-8 in Perl tests
- 42:40Handle invalid UTF-8
- 45:51Q&A on Unicode edge cases
