Schedule AI workloads efficiently on Nomad with node pools
summary
Deploys a Nomad cluster on AWS with a single Nomad server and client node pools by instance size, using Terraform to create a node pools layout (small public, large private, medium private) and attaching CSI/EBS volumes for persistence. Schedules Ollama-backed models (IBM Granite 3.3 on large, Granite 4.0 on medium) and a public Open WebUI frontend, wires service discovery, injects AWS credentials via Nomad variables, uses Nomad actions and nomad exec to create a pirate-tuned model, and explains autoscaling though it isn’t run.
AI-written summary · about
chapters (15)
- 0:00Introduction to workshop
- 1:47Workshop agenda overview
- 3:33Pirate problem scenario
- 8:10AI workload challenges
- 13:37Nomad solutions explained
- 15:05Hands-on lab introduction
- 22:35Setting up interactive lab
- 26:10Deploying infrastructure with Terraform
- 37:06Verifying Nomad cluster
- 39:35Deploying AI model with Nomad
- 43:04Deploying Open Web UI
- 47:28Testing AI chat functionality
- 51:44Creating pirate model variant
- 56:02Scaling cluster with new node pool
- 1:03:34Workshop summary and Q&A
related talks
30mAutoscaling with HashiCorp Nomad
27mNomad Best Practices for Reliable Deploys
44mNomad, orchestration made in HashiCorp
30mA beginner's journey: Scaling self-hosted platform infrastructure with Nomad
35mLiving the Nomadic life
33mAbstracting Nomad: Patterns for Deploying to Nomad at Scale
22mRainbow deployments with Nomad
23mMachine Learning Workflows with HashiCorp Nomad and Apache Spark