Govern Your Event Stream
summary
CROZ builds a small Java glue that reads Kafka audit logs and schema registry entries and creates topics, schemas and lineage in Apache Atlas in near real time; it also syncs Atlas tags into Ranger to enforce access policies. The tool named Cross Auditor registers new topics and schemas automatically (customers run from five servers to 100 servers and petabyte datasets) and so reduces missing metadata and surprise breakages while leaving compatibility checks to the registry.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 0:46Data governance motivation
- 4:20Customer story obstacles
- 5:27Data lineage with Atlas
- 8:23Data quality in Kafka
- 9:49Schema registry governance
- 14:03Data catalog features
- 17:55Real-time governance solution
- 19:45Cross Auditor Kafka integration
- 24:03Customer use cases
- 25:30Q&A
related talks
01h 00mEvolving Your Kafka Architecture With Schema Registry
25mFrom Guesswork to Governance: Bringing API Discipline to Data Streaming
15mAn Empty Kafka is a Sad Kafka
43mFrom Tower of Babel to Babel Fish: Evolving Kafka Architecture With Schema Registry
26mKafka in the Wild
41mKafka as a Platform: The Ecosystem from the Ground Up
41mKafka as a Platform: the Ecosystem from the Ground Up
46mKafka Streams at the Big Data Crossroads at Lightning Speed