~/talk

SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation

on YouTube 18m
Talkat USENIX ATC 2024 · 2024-07-10..07-12· [machine-learning, cloud-infrastructure]

speakers

name:YIFAN XIONG
links:none on file
name:Yuting Jiang
links:none on file
name:Ziyue Yang
links:none on file
name:Lei Qu
links:none on file
name:Guoshuai Zhao
links:none on file
name:Shuguang Liu
links:none on file
name:Dong Zhong
links:none on file
name:Boris Pinzur
links:none on file
name:Jie Zhang
links:none on file
name:Yang Wang
links:none on file
name:Jithin Jose
links:none on file
name:Hossein Pourreza
links:none on file
name:Jeff Baxter
links:none on file
name:Kushal Datta
links:none on file
name:Prabhat Ram
links:none on file
name:Luke Melton
links:none on file
name:Joe Chau
links:none on file
name:Peng Cheng
links:none on file
name:Yongqiang Xiong
links:none on file
name:Lidong Zhou
links:none on file

resources

video:https://youtu.be/mfNRzHHPIiE
slides:none on file

summary

SuperBench is a proactive validation system that runs a comprehensive benchmark suite (end‑to‑end and microbenchmarks), uses a Validator that learns benchmark criteria from cumulative distributions to flag defective nodes, and a Selector that balances validation time and incident risk by choosing minimal subsets. Simulations show up to 92% shorter validation time, 9% higher cluster utilization and 11% longer mean time between incidents; paper claims up to 22.61× MTBI improvement, and Azure deployment validated ~24,000 A100s and filtered 10.36% defects.

AI-written summary · about

chapters (10)

  1. 0:00SuperBench introduction
  2. 1:29Why AI incidents happen
  3. 4:23Redundancies masking failures
  4. 5:29Proactive validation
  5. 6:12What to validate
  6. 9:03Benchmark expectations
  7. 11:10When to validate
  8. 13:37How SuperBench works
  9. 14:50Evaluation results
  10. 16:51Real-world impact

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.