Manipulating Data in an Artificial Intelligence World
summary
AI training needs massively faster local storage and more memory bandwidth: GPUs account for 80% of system cost, so feeding them matters. A 100 TB transfer test shows replacing HDD arrays with SSDs cuts data-stage time by four days. Use NVMe SSDs (up to ~30 TB) for GPU servers, mainstream NVMe for inference, and SSD arrays to avoid GPU idle time. Memory speeds move from 4800 → 5600 → 6400 MT/s with roadmaps past 8800, non‑binary DIMMs like 24/48/96 GB reduce stranded capacity and 128 GB modules cost ~$1,500–$2,000 (Micron ~$1,200).
AI-written summary · about
chapters (8)
- 0:00Introduction and speaker intro
- 2:03AI impact on data center hardware
- 5:39Training vs inference hardware needs
- 9:29GPU server storage optimization
- 13:06External storage for AI workloads
- 16:41Memory trends for AI infrastructure
- 20:57High-capacity memory solutions
- 22:45Future AI hardware demands
related talks
39mUnlock the potential of your data: Optimize your data store architecture...
22mCurrent State of AI
58mKeynote: Enabling the AI Revolution: Next-Generation Memory Solutions for AI Computing(…)
24mHow to Eliminate the I-O Bottleneck & Continuously Feed the GPU While Training in the Cloud
40mDemystifying AI Hardware: NVIDIA Evolution and Costs
12mFlashNeuron: SSD-Enabled Large-Batch Training of Very Deep Neural Networks
16mCognitive SSD: A Deep Learning Engine for In-Storage Data Retrieval
12mOptimizing AI Data Infrastructure with F5 and