GPU Accelerated Containers on Apple Silicon with libkrun and podman machine
summary
Shows libkrun plugged into podman machine (via krunkit) to enable Apple GPU passthrough so a local 7B inference run (mistl/llama.cpp-style model on macOS Metal) offloads work to the M‑GPU instead of the VM CPU. Measured example: the same prompt takes ~12.5 minutes on Apple hypervisor at ~70% CPU in the VM versus ~1 minute with libkrun at ~5% CPU, demonstrating concrete GPU acceleration and usable VM integration.
AI-written summary · about
chapters (8)
- 0:00GPU containers on Apple Silicon
- 1:04Containers on Mac OS
- 2:28Podman machine and virtualization
- 4:33Hypervisor vs virtualization
- 5:39What is libkrun
- 7:49Podman with libkrun for heavy apps
- 8:54Live demo: performance comparison
- 12:31Future work and contributions
related talks
34mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
12mKeynote: New Advances for Cross-platform AI Applications in Docker
06mProject Lightning Talk: K8s issue #52757: Sharing GPUs Among Multiple Containers
09mWhy I ditched cloud transcoding and set up a mac mini render farm
27mRunning Containerd and k3s on MacOS
23mLLMs on the Edge: The Future of On-Device Intelligence
13mKeynote: Blurring the Lines: The Continuum Between Containers and VMs
05mContainerization Beyond CPUs – A Kubernetes-based Serverless Pl...