Build your own LLM on Cloud
summary
Architecture shows deployment of a base Gemma 7B model for text completion and a fine-tuned Gemma using the Databricks Dolly dataset (about 15k examples) to enable chat behavior. Fine-tuning runs on a GPU VM pulling checkpoints from and writing to GCS, invoked by commands such as kubectl apply -f fine-tune-job.yaml, then deployments run on GKE behind internal load balancers with a Cloud Run LangChain front end and Hugging Face TGI model servers.
AI-written summary · about
chapters (6)
- 0:00Meet Ali
- 0:31Demo architecture
- 1:29How to fine-tune LLMs
- 2:07How to deploy LLMs on GKE & Cloud Run
- 2:51Chatting with the LLMs
- 4:14Wrap up
related talks
01h 15mWorkshop: Working with Gemma and Open LLMs on Google Kubernetes...
09mUsing and fine-tuning Gemma 3
44mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model
27mYes You Can Run LLMs on Kubernetes
12mDemo: Building cloud-native, AI-powered applications with GKE
21mFrom 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents
43mRunning open large language models in production with Ollama and serverless GPUs
39mFlutter Project with Gemma: A State-of-the-Art Open Model