Efficient Performance-Aware GPU Sharing with Compatibility and Isolation through...

on YouTube ↗14m

" summary

KRYPTON implements kernel-space command-buffer interception and a spatio-temporal orchestrator on A100 and 4090 hardware, using offline profiling, CPU tokens, and command-buffer locking (read-only + segfault signal handler) to enforce precise time slices. It packs 20 apps onto four GPUs, reduces GPU count by 32.1% versus SOTA, and uses a feedback controller to meet per-app throughput targets while keeping stable utilization at 25 and 75% across DL training, inference, and graphics workloads.

AI-written summary · about

" chapters (12)

1. 0:00GPU sharing challenges
2. 1:23Why GPU sharing is necessary
3. 2:50Limitations of existing solutions
4. 4:15Kernel space interception key
5. 6:19Krypton architecture
6. 7:21Spatial temporal orchestration
7. 8:45Performance control mechanism
8. 9:46Kernel space interception details
9. 10:27Evaluation setup
10. 11:08Performance results
11. 12:33ResNet 50 performance analysis
12. 12:54Conclusion and future