Orchestrating multi-cloud data pipelines with
summary
Control‑M claims to abstract multi‑cloud pipeline complexity and shows concrete integrations: Kafka → Lambda/micro‑batch → S3 → Athena → SageMaker with Tableau/Looker visualization, plus automated restarts (three attempts) and impact analysis. A customer case ingests over a trillion records/day to surface the top 1% most harmful malware. Orchestration supports Control‑M Automation API (REST), JSON jobs‑as‑code and CI/CD (Jenkins), offers on‑prem and Helix SaaS, enforces naming and governance checks before production.
AI-written summary · about
chapters (8)
- 0:00Introduction to session
- 1:04Data investment and challenges
- 6:19Key stages of data pipelines
- 9:48Control-M orchestration solution
- 11:35Collaboration and automation gap
- 13:22Jobs as code approach
- 16:58Malwarebytes use case
- 20:09Q&A and demo preview
related talks
34mWorkflow Orchestration Embedded in your DevOps Tool Chain
37mDeploying to the Cloud Multiverse! IBM Multicloud Manager
36mMLOps from the edge to on demand visualization
56mCloud compliance journey: Compliance and audits
05mYou scream for microservices orchestration; I scream for batch...
26mRunning Secure AI/ML Experiment in a Unified MLOPS-DevSecOps Pipeline
53mExperience with an end-to-end MLOps pipeline: Practical insights and 3D architectural perspectives
28mMulti-cloud Cost Management Platform with FOCUS