Working with Multilingual Data in SQL Databases through Erlang
summary
A SQL virtualization engine written in Erlang translates Teradata and Oracle queries to BigQuery Azure or Postgres and proves that correct Unicode handling requires concrete choices not vague promises The team fixes Cyrillic corruption by consolidating string logic into one module using the Erlang unicode library ODBC drivers terminals and Terraformized test stacks reveal where encoding flips happen They adopt UTF-8 everywhere, preserve metadata about original encodings, rely on hex dumps as hex is truth, and deliver reliable read write of multilingual data.
AI-written summary · about
chapters (11)
- 0:00Unicode and Erlang intro
- 1:50System overview
- 3:40Cyrillic data challenges
- 7:12Unicode and mangling
- 12:16Erlang string encoding
- 17:53UTF-8 as standard
- 19:01Debugging learnings
- 21:07Metadata preservation
- 24:42Testing and config pitfalls
- 25:44Hex versus tools
- 26:49Q&A
related talks
20mMultilingual Data Processing with GraalVM
32mBecause SQL is everywhere but you don't want to use it
36mBeyond dbt: Modern SQL Transformation and Lineage with SQLGlot and SQLMesh
20mStoring strings in SQL Server and Azure SQL DB
32mHave a SQL and Eat it Too!
21mGraalVM & Database Multilingual Engine
47mMulti Database Colaboration and Extract - Transform - Load with HTSQL
27mTroubleshooting the Performance of a Large Erlang System