Fluid: Kubernetes Native Distributed Dataset Orchestrator and Accelerator
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2021 · on Friday, 2021-12-10· at 13:15· Kubecon + CloudNativeCon 演讲厅, 虚拟会议 (Virtual)·
summary
Fluid is an open-source CNCF sandbox project that orchestrates and accelerates distributed datasets for big data and AI workloads on Kubernetes. It addresses performance and scalability problems when accessing disaggregated storage (S3, Google Cloud Storage, HDFS) by providing a Kubernetes-native architecture and features to speed training and serverless computing; the talk covers Fluid’s motivation, architecture, recent features, user adoption, roadmap and how contributors can get involved, plus a demo of accelerating serverless workloads with Fluid.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:47Agenda
- 1:31Data-intensive Applications Are Trending on Kubernetes
- 3:25Journey of A Data-driven Company
- 6:02Technical Challenges Blocking Faster Adoption
- 6:45st Challenge: Complicated Data Storage
- 8:39nd Challenge: WO Bottleneck
- 12:01rd Challenge: Locality-ignoring Scheduling
- 14:39What is Fluid, indeed
- 17:55Fluid Architecture
- 21:41A Quick Start on Fluid: (1) Create a "Dataset"
- 23:56A Quick Start on Fluid: (2) Run Apps with Data Affinity
- 24:42Quick Start on Fluid: Auto-scale Dataset
- 25:38Demo
- 28:53Roadmap
- 30:19Plan
- 32:52Community & Join Us
related talks
29mFluid - Build Data Orchestration in Kubernetes
05mProject Lightning Talk: Fluid Data Anyway, Data Anywhere, Data Anytime
45mFrom Zero to Infinity: How AI-Powered Hedge Fund Build Cloud-Native AI Platfo...
37mComputational Fluid Dynamics (CFD) Analysis With Kubernetes, Kubeflow, And OpenFOAM
34mKubeflow: Multi-Tenant, Self-Serve, Accelerated Platform for Practitioners
30mCloudflow: Spark, Flink, and Akka Working together on Kubernetes
09mLightning Talk: Fluentd at Scale; Keys to successful Logging Syndication
48mKubeFlow BoF