KernelCI New Generation
summary
KernelCI New Generation reindexes KernelCI data into an ELK stack to make logs and test results searchable and visual, ingesting builders, LAVA boot JSON and test JSON and duplicating streams into Elasticsearch, Logstash, Kibana. Each LAVA log line becomes one document per log line so searches return exact lines, daily indices enable retention and fast deletion, and the pipeline keeps 14 day retention. Implementation uncovered Logstash JVM and memory issues (single documents ballooning to 10+ GB), and dashboards show about 3% failing builds among hundreds of thousands of builds.
AI-written summary · about
chapters (12)
- 0:00Introduction to KernelCI
- 1:28Problems with old frontend
- 3:34How KernelCI currently works
- 4:39Introducing new data pipeline
- 5:20Challenges with Elasticsearch
- 6:24Kibana dashboards overview
- 11:20Live demo screenshots
- 13:49Boot visualization features
- 16:44Test visualization features
- 18:13Log search functionality
- 21:24Next steps and evolution
- 22:33Q&A on data indexing
related talks
01h 00mELK Stack - All You Need for Search In Your Applications
30mData @Scale Seattle- Sundaram Narayanan
15mLinux Testing made better with DATA
42mGoing Elastic
23mLLMs on the Edge: The Future of On-Device Intelligence
15mA Community Cache with Complete Information
41mCross Industry Demands and Collaboration Opportunities
09mLogCrisp: Fast Aggregated Analysis on Large-scale Compressed Logs by Enabling...