The Secure Metamorphosis: Streaming Logs with Kafka and TLS
summary
Kafka handles 5 billion messages/day into topics and partitions while TLS is added for transport; builders implement mutual TLS plus SASL (plain/SCRAM/Kerberos) and Kafka ACLs, and back up disks with encrypted EBS for encryption-at-rest. ZooKeeper lacks TLS so operators isolate and harden it. Moving from Java 1.8 to Java 1.9 improves throughput, and the Go client Sarama limits cipher choices. PKI management and certificate revocation prove painful; the secure setup works but is operationally complex to automate.
AI-written summary · about
chapters (9)
- 0:00Introduction and agenda
- 1:07Data and Kafka basics
- 6:25Why secure Kafka
- 8:10Kafka defaults and risks
- 10:42Transport encryption with TLS
- 18:11Client authentication
- 21:25Authorization and ACLs
- 27:48Lessons learned and resources
- 29:11Q&A on GDPR and closing
related talks
41mReturn of Zero-Copy: How We Hooked up kTLS to Apache Kafka
41mKafka Security - A brief overview of its history ... #bbuzz
45mDeep Dive into Data Streaming Security
56mDeep dive into data streaming security
27mTGI Vault: Securing Kafka with Vault PKI
43mAttacking (and Defending) Apache Kafka
27mEnd-to-End Encryption for Streaming Data Pipelines
07mDescribing Kafka security in AsyncAPI