Streaming with Structure
summary
Shows how schemas stop consumer breakage by centralizing event blueprints in a registry instead of embedding them in every message, and compares formats and transport choices: Apache Avro (most common with Kafka), JSON Schema, and Protocol Buffers; demos how to carry schema identity via the original magic byte or via record headers, and how serializers/deserializers, Kafka Connect converters and Kafka Streams serdes must be configured. Mentions Apicurio and Confluent registries and that Avro supports generic records without codegen.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:53Barista example
- 8:49Using JSON
- 10:20Adding structure with schemas
- 13:20Schema registry
- 14:07What format to use?
- 19:33Which schema to choose?
- 27:02Serdes and converters
- 34:41Apicurio registry
- 36:06Confluent schema registry
- 36:51Summary
- 37:28Outro
related talks
25mFrom Guesswork to Governance: Bringing API Discipline to Data Streaming
23mEver Changing Data Model Schema Management For The Future
39mMoving Beyond Moving Bytes
01h 00mEvolving Your Kafka Architecture With Schema Registry
43mFrom Tower of Babel to Babel Fish: Evolving Kafka Architecture With Schema Registry
03h 00mStream Processing, Going Beyond Pub/Sub with Kafka Streams and...
31mEvent Streaming with Kafka Streams
01h 00mKafka as a Platform: the Ecosystem from the Ground Up