Keynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
summary
NVIDIA claims 500 open-source projects and notes over 100 million RTX PCs as a target install base. The stack highlights concrete tools: run pandas unchanged with RAPIDS accelerator to GPU-boost ETL, Megatron-LM for multi‑thousand‑GPU training (research since 2017), Nemo toolkit for ASR/TTS/LLM fine-tuning, Triton inference server with 30+ backends, RAFT for GPU-accelerated vector search, MRAW sparse MoE with 47B parameters routing ~13B per token, and Nemo Guardrails for safety.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:47Nvidia has over 500 open source projects
- 1:30The LLM Stack
- 2:34Open Source Tools
- 2:56AI Foundations Models
- 4:06Data Processing
- 5:18Rapid
- 5:37Spark
- 5:51Megatron LM
- 7:07Nemo Toolkit
- 7:41Triton
- 8:21Backends
- 8:55Rag
- 9:26Raft
- 10:46Rails
- 11:15Outro
related talks
12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
13mKeynote: To Nemotron and Beyond: Building the Open Source AI Ecosystem
28mDeveloper Essentials for the New Era of AI
11mKeynote: Open Source Stack Empowers the AI Era
24mKeynote: Next-generation frameworks for Large-scale AI
52mSimplifying the adoption of generative AI for enterprises
16mKeynote: An Open-Source Stack for AI Compute: Kubernetes + Ray + PyTorch + vLLM
13mKeynote: How We Save $900 per Day with Self-Hosted AI: Building Scalable Local LLM Infrastructure