ZML: ML framework for Zig
summary
ZML compiles Zig model code via MLIR/XLA and Bazel into production inference artifacts that run on NVIDIA, AMD (ROCm) and TPU, with a rebuilt Torch loader and zero Python in the pipeline. Axis tagging removes most transposes and reshape bookkeeping, kernel fusion plus autotuning produce optimized kernels, and Bazel-driven cross-compilation yields 90% size reduction and reproducible cross-accelerator binaries demonstrated from MacBook to Docker images.
AI-written summary · about
chapters (10)
- 0:00ZML introduction and training vs inference
- 1:30Why build another ML framework
- 2:34ZML’s four-component stack
- 3:20ZML focus: inference and Zig frontend
- 4:48Writing modular ML code
- 5:51Tagging for cleaner tensor ops
- 7:16OpenXLA for cross-platform compilation
- 8:40Basil runtime and deployment
- 10:26MLOps with ZML cross-compilation
- 11:33ZML traction and future plans
related talks
32mTaming the Machine: Basics of ML Models Training and Inference Optimization
36mTinyML at the Edge: Deploying and Optimizing AI Workloads on Zephyr RTOS
23mWhy MLX
17mEmpower ML Framework on RISC-V by Backend Graph Compilers
14mKeynote: Leveraging Zephyr and ML to Bring Smart Devices to Market, Faster
11mUniversally Deploy Large-language Models via ML Compilation
06mML for Systems and Systems for ML
10mMachine Learning for mobile