How AI Infrastructure Differs from the Ordinary
summary
The talk compares AI infrastructure to traditional infrastructure, focusing on challenges from large models, latency, and autoscaling. It draws on experience at LetsEnhance/Claid (image enhancement and background generation) and outlines practical views and proposed fixes for running AI systems, aimed at mid/senior engineers already working with or curious about operating AI workloads.
AI-written summary · about
chapters (14)
- 0:00AI infrastructure overview
- 4:01Introduction
- 5:26AI vs ordinary infra
- 8:40AI model nuances
- 15:18Nvidia Triton for inference
- 20:39Inference types & challenges
- 25:23Auto-scaling in AI
- 33:18Reducing AI costs
- 36:10Model delivery & storage
- 37:58AI as black box
- 38:39Chatbot state retention
- 40:31DirectStorage & load speed
- 43:28Cheap cloud providers
- 45:38Summary
related talks
47mReference Architecture of AI in the Cloud
54mArtificial Intelligence seen from the software development lifecycle perspective
19mComputing AI - Obstacles and Implications
46mIntroduction to AI engineering
32mFrom Intelligence to Autonomy - Building the Infrastructure for the AI Workforce
10mApproach to AI at a large scale
17mChallenges and takeaways of managing AI workloads on cloud
17mAI TechOps: Data and Infra for AI Foundation to Scale and Manage 1000s of Cohesive Agents