Data Integration in the World of Microservices
summary
Zalando explains why isolated service databases break ETL and how PostgreSQL’s logical replication actually streams row‑level changes out of separate AWS accounts into a central system. The company runs an open‑source prototype on top of its STUPS.io platform that captures changes, creates consistent snapshots, and materializes them for Data Lake, operational stores and KPI pipelines. The context: nearly 1,000 engineers and a plan to add 1,000 more in 2016; concrete tools, replication plugin names, and integration patterns are shown rather than high‑level exhortation.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:08Solando’s growth and tech
- 3:57Microservices transition challenges
- 6:08Troops infrastructure and compliance
- 8:34Microservice data integration
- 10:46Event bus and consistency
- 12:12Logical replication tools
- 15:30Business intelligence data
- 16:56Psyche data platform
- 19:55Ensuring data correctness
- 21:32Real-time with Flink and Spark
- 23:21Data lake for accessibility
- 25:06Microservice prototyping with Swagger
- 27:17API guild collaboration
- 31:48Event-driven architecture
- 34:18Databases in Docker
- 36:54Q&A
related talks
22mData integration in the world of microservices
48mOnline OLTP Computing and Traffic Governance as a Service for True Digital Transformatio
55mThe hardest part of microservices: Your Data
33mTransitioning from Debezium ElasticSearch to Postgres for Streamlined Resilience
38mFrom Monolith to Microservices at Zalando
01h 05mMigrating to Microservice Databases: From Relational Monolith to Distributed Data
31mHarness the Power of a Single Database for Microservices
20mScaling Microservices and Distributed Databases in a Multi-Cloud World