Efficient AI with Python: running AI models on CPUs
summary
Demonstrates running inference on CPUs using OpenVINO with Python, exporting models to an optimized format and letting the runtime re-optimize per-processor; uses small fine-tuned models and quantization to trade raw accuracy for speed, reporting a quantized 14B model deployed with comparable task-specific performance to GPU and that 1–1.5B models run on ~8 GB RAM for 1–1.5B models; first-token latency and cost savings drive design decisions.
AI-written summary · about
chapters (9)
- 0:00Introduction and speaker background
- 3:37Key requirements for AI adoption
- 6:51CPU vs GPU for AI inference
- 9:04Choosing efficient AI models
- 13:05OpenVINO framework demo
- 15:39Production pipeline and community contributions
- 17:06Performance results and scalability
- 19:16Real-world use cases and business value
- 21:25Q&A session
related talks
23mOptimize your network inference time with OpenVINO
30mCloud? No Thanks! I’m Gonna Run GenAI on My AI PC
19mOptimizing CNN Model Inference on CPUs
13mAccelerating AI algorithms
16mUnleash the superpowers of the AI PC
24mTools & Techniques for a cost optimized Analytics & AI pipeline
20mComputer Vision at the Edge with OpenVINO
42mAI Everywhere– accelerate your development from PC to cloud