Demo: Building cloud-native, AI-powered applications with GKE
summary
Fine-tunes a Gemma 2B model on a 5,000‑row SQLContext dataset inside a Docker image, runs training on 8 L4 GPUs in GKE (real run ~33 minutes, effective 625 steps across GPUs), combines LoRA weights into a single model and pushes it to Hugging Face as MofiCode/Gemma-2b-sql, then deploys via a TGI/Triton YAML on GKE to serve SQL-to-SQL queries over HTTP with options shown for GPU and TPU serving.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:35Why use Kubernetes
- 2:58GKE autopilot
- 4:06Finetuning Gemma
- 8:30GKE Guides
- 9:01Serving Open Models
related talks
01h 15mWorkshop: Working with Gemma and Open LLMs on Google Kubernetes...
27mYes You Can Run LLMs on Kubernetes
05mBuild your own LLM on Cloud
32mRunning and Fine-tuning Open Source LLMs on Google Kubernetes Engine
09mUsing and fine-tuning Gemma 3
43mRunning open large language models in production with Ollama and serverless GPUs
44mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model
32mGPU Machine Learning From Laptop to Cloud