Managing CI Across Multiple Farms for GPU Testing
summary
Mesa3D CI runs on a self-hosted GitLab CI backed by multiple farms (LAVA, Valve-style containers, bare metal) with ~200–250 jobs and a goal: 20-minute pre-merge turnaround; tests run on Debian/Fedora/Alpine images, using rootfs for performance or containers for developer reproducibility. Parallelism splits long suites across boards to keep individual runs under ~10 minutes, a home LAVA farm ran 1,682 jobs last month, infrastructure uses SSH fallback for flaky serial, Wi‑Fi power switches, NFS/TFTP, seven kill switches, and prevents real regressions. LAVA + GitLab CI rootfs vs container tradeoff
AI-written summary · about
chapters (11)
- 0:00Start
- 0:51Overview of today's talk
- 2:08Before we start!
- 2:47Build + Lint
- 3:59Testing without HW
- 5:26DUT-CI Farms
- 9:29Level 3: DUT Environment
- 11:44Level 3: DUT Stability
- 16:55Level 3: Conclusion
- 18:52running your own LAVA farm
- 29:07What's coming next
related talks
38mGrowing a Lab for Automated Upstream Testing: Challenges and Lessons Learned
39mQuickly Test Your Kernel with GitLab CI
37mAutomated Testing and Continuous Integration/Delivery: Lessons Learned
29mHow Hard Is It to Test the GPU Driver?
01h 05mFedora CI Meetup
37mKernel CI – How Far Can It Go?
26mLAVA + OpenQA = Automated, Continuous Full System Testing
29mTesting, a Journey from Testing Kernels to Testing Debian and Yocto