How does Docker run machine learning on specialized AI
summary
Explains how Docker exposes GPUs and AI silicon by inserting a pre-start hook via a custom runtime that calls vendor libraries (example: libnvidia-container for NVIDIA or the oci-neuron shim for AWS Inferentia), leaving drivers on the host and mapping /dev into the container. Demo shows docker run --runtime=nvidia --gpus=all enabling nvidia-smi inside a container, and an AWS demo revealing one neuron device with four cores. Covers Jupyter-in-container development and Kubernetes device-plugin/daemonset deployment patterns.
AI-written summary · about
chapters (8)
- 0:00Introduction to ML hardware
- 2:22Challenges in ML stacks
- 6:07Docker hardware integration
- 9:35The role of custom runtimes
- 15:10NVIDIA GPU implementation
- 22:01AWS Inferentia example
- 27:45Development and reproducibility
- 32:03Orchestration and conclusion
related talks
24mDocker for Machine Learning!
30mRunning AI Workloads in Containers and Kubernetes
35mState of Containers and the Convergence of Hyperscale, BigData and HPC
31mNavigating Containers: AI Powered by Docker
35mAccelerate deep learning in the cloud with custom ML environments
32mGPU Machine Learning From Laptop to Cloud
06mLightning Talk: Deploying AI Solutions on the Edge with Containers
28mEnd to end AI developer journey with containerized assets using Intel DevCatalog, Intel DevCloud