Kafka, DevOps...And Resilience for All
summary
Shows how Kafka cluster ops are turned into repeatable IaC workflows: uses Bash, a terraform provider or a Kafka-based kafka config manager to declare topics, partitions, ACLs and TLS; demonstrates secrets via Vault or sealed Bitnami secrets for k8s and a sidecar pattern for cert delivery. Covers backups (MSK, MirrorMaker, Kafka Connect to S3) and a real recovery hazard where restoring data without offsets causes runaway reprocessing. Emphasizes log offsets and partitions and monitoring lag.
AI-written summary · about
chapters (20)
- 0:00Intro
- 2:16Kafka - what's the difference?
- 3:49Key concepts
- 7:30What can I DevOpsify?
- 10:40What are the options?
- 11:21Bash
- 12:17Terraform
- 13:33Kafka config manager
- 14:47Secrets management
- 16:48In an ideal world
- 18:33In a k8s-native world
- 20:27Backups
- 21:54Backup options
- 25:37What about application parameters?
- 31:20Monitoring
- 35:18Monitoring tooling
- 36:33Tooling - commercial
- 37:14Tooling - open source
- 38:01Key takeaways
- 40:12Outro
related talks
02h 48mKafka: Map of Pitfalls for Developers and Ops
33mKafka Operator: Managing and Operating Kafka Clusters in Kubernetes [A]
29mKafka on Kubernetes: Does it really have to be the hard way?
44mDiving into the World of Apache Kafka @ Michelin: The Balance Between Dev and Ops
23mWhen Kafka Goes Cloud Native: Observability That Actually Works!
33mApache Kafka Devops with Kubernetes and Gitops
26mProduction Grade Kafka on K8s
30mKafka on Kubernetes: From Evaluation to Production at