Cloud Foundry and ODPi Working Towards Enabling Cloud-Native, Data-Driven Applications

on YouTube ↗31m

" summary

ODPi provides a runtime spec and a BigTop-based reference implementation and the prototype wires BOSH to BigTop to deploy a Spark+HDFS+Zeppelin cluster on AWS. An orchestrator above BOSH generates manifests and aims for one-click deployment, spinning a BOSH director and nodes; it works end-to-end but requires a custom agent package and out-of-band RPM/DEB access. BOSH offers robust IaaS abstraction and self-healing; gaps include cumbersome BOSH-only packaging, difficulty expressing complex HDFS/spark topologies, and nascent links for richer config relationships.

AI-written summary · about

" chapters (6)

1. 0:00Introduction and speakers
2. 3:09What is ODPi and its role
3. 6:59Data-driven applications on Cloud Foundry
4. 14:27Prototype architecture and approach
5. 23:02Lessons learned and improvements
6. 27:59Q&A session

" related talks

• 14mDelivering the enterprise data cloudAnthony Huinker · AWS re:Invent 2019
• 20mData in the Apache "big data" ecosystemKonstantin Boudnik · Data By the Bay 2016
• 32mBuilding Data-Driven IoT Apps on Cloud Foundry - Dormain Drewitz, PivotalDormain Drewitz · Cloud Foundry Summit Europe 2017
• 30mPlatform Oriented Data Architecture for Acceler...Vladimir Bacvanski · Global Artificial Intelligence Conference 2020
• 26mModern Data OrchestratorsRiccardo Amadio · Community Over Code EU 2024
• 39mOrchestration of Big Data systems: A survivor's guideKamil Owczarek · JDD 2018
• 45mEmbracing Big Data Workload in Cloud-Native Environment with Data LocalitySammi Chen, Xiaoyu Yao · KubeCon + CloudNativeCon China 2019
• 01h 32mCloud Native Data Flow OrchestrationMark Fisher, Patrick Peralta · SpringOne2GX 2015