Data Mesh and Data API
summary
Claims that event streaming with Confluent and Kafka turns slow, batch data into real-time data in motion, using source connectors plus a stream-processing engine (ksqldb-style transforms) for real-time ETL, enrichment and routing. Example: Grab uses the platform and cuts fraud to 0.2% fraud rate while speeding development. Architecture supports continuous streams or REST snapshots, a stream catalog and lineage for self-service, and federated governance to treat data as a first class product across domains.
AI-written summary · about
chapters (11)
- 0:00Data Mesh and APIs intro
- 1:04Event streaming basics
- 3:09Batch to real-time data
- 6:21Database limits
- 8:09Data in motion and Kafka
- 12:19Streaming system traits
- 15:28Data Mesh principles
- 17:15Domain-driven decentralization
- 19:47Self-service platforms and governance
- 21:32Kafka for Data Mesh
- 22:14Kafka and Data Mesh setup
related talks
40mA Data Mesh Of Streams
39mData Streaming? I don't even know her
53mData Mesh in Practice - Assuring Data Quality at Scale
44mData Mesh
06mJay Kreps interview
29mData Mesh Streaming Patterns: A Real-time Perspective
26mLeveraging Data in Motion in a Cloud-first World
28mTransforming the Financial Services Industry with Event Stream Data