Cloud-native ETL with Java Quarkus, Kubernetes, and Jib Container Builder
summary
Cloud-native ETL architecture with Java Quarkus, Kafka/Redpanda, and Jib shows a concrete pipeline: GitHub Actions builds artifacts, ArgoCD deploys immutable images, Jib builds container images without a Docker daemon and tags them by exact hash, Quarkus can compile to a native binary, and Kafka stores offsets so pipelines resume precisely after failure. Kafka Streams apps get compiled and deployed at runtime, about 50 no-code transformations exist today, and connectors are moved out of shared clusters to run as sidecars for self-containment and predictable resource limits.
AI-written summary · about
chapters (8)
- 0:00Introduction and speaker intro
- 1:25Problem with current ETL practices
- 3:11GitOps and CI/CD in ETL
- 5:18Cloud-native principles for ETL
- 9:10Practical implementation with Quarkus and Jib
- 16:21Current ecosystem and future vision
- 18:50StreamZ and Kafka Connect overview
- 23:47Q&A session
related talks
53mMeet Quarkus
01h 01mTaming Kubernetes Streamlining Inner loop Development for Distributed Systems
46m5 Ways to Make Java Development and Kubernetes Spark Joy With Quarkus.
46mKubernetes. From 0 to Production-Grade with Java.
49mKubernetes. From 0 to Production-Grade with Java
36mCrafting Intelligent Cloud Native applications with Quarkus
47mJava App Development in the Cloud: MicroProfile, Quarkus and Cloud.
40mYour Java Application Kubernetes Native Ready: A Live Experience