Reliable K8s Resource Submission & Bookkeeping
Talkat KubeCon + CloudNativeCon Europe 2025 · on Wednesday, 2025-04-02· at 17:00· Level 1 | Hall Entrance N10 | Room H·
summary
Bloomberg describes a concrete system: a user API writes actions to a source‑of‑truth DB and a message stream (Apache Kafka) while a submitter consumes runnables with retry and deadline verification to place jobs into clusters; deployables sync from a highly available PostgreSQL source via a Kubernetes watcher. An in‑cluster producer publishes creates/updates/deletes plus a periodic cluster snapshot so the consumer can remove “zombie” inventory records, restoring consistency and low‑latency reads.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 4:13High-level architecture
- 7:28Separating API and submitter
- 9:41Post-deployment status tracking
- 11:52Persisting historical transactions
- 14:46Handling zombie records
- 17:41Challenges and solutions
- 19:07Q&A
related talks
39mBuilding Distributed Systems with Kubernetes
37mHow Bloomberg Creates a Resilient Data Analytics Platform Using Karmada
52mHeterogeneous pipeline processing with Kubernetes and Google Cloud Pub Sub
31mEvent driven communication with Kafka on Kubernetes
35mKubeArchive: Don’t use Kubernetes as a Database
40mThe Good, the Bad, and the Ugly of Managing Production Systems with Kubernetes
31mManaging Multi-Cloud Apache Spark on Kubernetes
41mKafka as a Platform: The Ecosystem from the Ground Up