Using Machine Learning to find Linux bugs

on YouTube ↗48m

" summary

Claim: flaky CI failures are real bugs and can be surfaced automatically. Pipeline ingests logs from 2.5 million tests over 90 days, sanitizes lines, uses TF‑IDF then computes pairwise similarity with Normalized Compression Distance, groups with DBSCAN clustering, visualizes with MDS, and retries suspected flakes. The system files bugs for the largest clusters, finds kernel and Cockpit races that were reproducible, and runs on CPU Python/Scikit‑learn — it works in practice.

AI-written summary · about

" chapters (4)

1. 0:00Start
2. 13:24Normalize Compression Distance
3. 15:25Basic Machine Learning Techniques
4. 30:11What's Next

" related talks

• 38mApplying machine learning to software development to reduce bugsNitin Borwankar · PyData San Francisco 2016
• 45mUsing Machines to exploit MachinesG. Barnhart Magen, E. Caltum · Hack In Paris 2019
• 47mBug prediction based on your code historyAlexey Tokar · JavaDay Ukraine 2017
• 50mEffective test scrubbingMarius Liaaen, Carl Martin Rosenberg, Thomas Hanssen Nornes · NDC TechTown 2017
• 50mPractical Applications of ML in Software TestingMesut Durukal · NDC Sydney 2020
• 35mBetter C++ using Machine Learning on Large ProjectsNicolas Fleury, Mathieu Nayrolles · CppCon 2018
• 07mFinding Bugs in Deep Learning ProgramsFoutse Khomh · Strange Loop 2022
• 17mAnnotating Log Data For Machine LearningJulia Li · All Things Open 2023