Engineering Scalable ML Infrastructure for Modern Apps
summary
Claims that matter: concrete patterns for scaling—distributed data parallelism for multi‑GPU training, GPU scheduling and container isolation for resource governance, and feature store reuse to avoid data drift. The talk outlines measurable tradeoffs (training versus millisecond inference), tool classes (container orchestration, stream processors, NVM storage), mitigation tactics (batching inference, checkpointing for fault tolerance), and compliance hooks (role‑based access, GDPR/HIPPA) rather than high‑level platitudes.
AI-written summary · about
chapters (16)
- 0:00Introduction and Speaker Background
- 0:23Overview of AI Native Platforms
- 0:56Infrastructure Imperatives for AI
- 1:55Understanding AI Workload Characteristics
- 2:44Data Pipelines in AI
- 3:30Infrastructure Patterns for Scalable Model Training
- 4:15Resource Management Challenges
- 4:50GPU and CPU Optimization
- 5:32Data Pipeline Architectures
- 6:12Observability and Monitoring for AI
- 6:50Real World Implementation Patterns
- 7:37Performance Optimization
- 8:13Security and Governance in AI Platforms
- 8:50Future Trends in AI Infrastructure
- 9:30Building the Roadmap for AI Innovation
- 9:56Key Takeaways and Conclusion
related talks
30mEssential Ingredients in Scaling Organizations for ML
41mInfrastructure for ML Applications
21mHarnessing Microservices for Scalable, Fault-Tolerant ML Systems
44mScaling ML from 0 to millions of users
33mBuilding Resilient (ML) Pipelines for MLOps
19mBuilding Resilient ML Cloud Integrations
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
41mPanel: AI Infra Best Practices: Enterprise Do’s and Don’ts