Automated Testing and Continuous Integration/Delivery: Lessons Learned

on YouTube ↗37m

" summary

Built a CI/CD pipeline that uses GitLab CI for SCM hooks, Jenkins for nightly Yocto Project image builds, Artifactory for binaries and OTA integration, and LAVA to run automatic regression tests on a board farm (47 devices, shelves up to seven carrier boards, ~100 modules tested per run). Claims: full flash-from-recovery tests on real hardware, shared sstate cache over NFS, primed builds to speed throughput, and resilience engineering because 99% reliability breaks trust. Operation requires ongoing maintenance and ~3–4 dedicated engineers.

AI-written summary · about

" chapters (28)

1. 0:00Introduction
2. 1:05Automated testing goals
3. 2:31Infrastructure tools
4. 5:02Code review processes
5. 6:51GitLab and Jenkins demo
6. 8:15Build and artifact management
7. 10:44Artifactory and reports
8. 11:29LAVA board farm setup
9. 13:40LAVA test execution demo
10. 15:09Hardware shelf controller
11. 16:35Loopback testing interfaces
12. 18:04Scaling lessons learned
13. 19:49Reliability strategies
14. 21:35Infrastructure as code
15. 22:39Team collaboration
16. 23:23CI/CD maintenance
17. 24:30Q&A: Dual build systems
18. 25:37Q&A: CI caching strategies
19. 27:24Q&A: Test infrastructure resources
20. 28:48Q&A: Bug reporting upstream
21. 29:50Q&A: Build state cache sharing
22. 30:13Q&A: Hardware simulation
23. 31:16Q&A: OTA testing automation
24. 32:45Q&A: Yocto state cache
25. 33:28Q&A: Jenkins alternatives
26. 34:54Q&A: LAVA re-init approach
27. 36:00Q&A: GitLab CI for Yocto
28. 36:43Q&A: Single-person Kubernetes CI

" related talks

• 44mAutomation beyond Testing and Embedded System ValidationJan Luebbe · Open Source Summit Europe + Embedded Linux Conference Europe 2017
• 48mLiving on Master: Using Yocto Project, Jenkins and LAVA for a Rolling ReleaseTim Orling · Embedded Linux Conference + OpenIoT Summit North America 2018
• 26mLAVA + OpenQA = Automated, Continuous Full System TestingLaurence Urhegyi, James Thomas · Open Source Summit + Embedded Linux Conference + OSPOCon 2021
• 38mAutomating and Managing an IoT Fleet Using GitMatthias Lüscher · Embedded Linux Conference Europe 2022
• 40mSecure by Default: Building an AI-Augmented, OSS-Powered Reusable CI/CD PipelineJenisten Xavier · Open Source Summit India 2026
• 53mOur CI/CD in Robot Testing at the Edge of Software and HardwareAlexey Boole · DelEx Conference 2020: DevOps & Test Automation
• 30mCI/CD - the game where all win!Admir Šatara, Ammar Karabeg · Bosnia Agile Day 2018
• 29mLarge Scale Deployments for Automated Kernel TestingDave Pigott · Open Source Summit Europe & ELC + OpenIoT Summit Europe 2018