Making Apache Spark, Apache Mahout, Kubeflow, and Kubernetes Play Nice
summary
Shows how Apache Spark and Apache Mahout can distribute very large matrices across many Kubernetes pods to work around per-node memory limits, and how Kubeflow is used to package and publish the end-to-end pipeline for reproducibility. Describes applying this stack to denoise DICOM lung images from COVID patients so the processing can be repeated by clinicians or researchers in resource-limited environments.
AI-written summary · about
related talks
36mTower of Babel: Making Apache Spark, Kubeflow, and Kubernetes Play Nice
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
21mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
26mBuilding Apache Spark Clusters on K8s for AI Workloads
46mDeploying scalable Machine Learning models with Kubernetes
39mOperating Deep Learning Pipelines Anywhere Using Kubeflow
29mScaling Apache Spark on Kube to Apple Scale