Netcastle: Network Infrastructure Testing At Scale
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
Netcastle provides CI/CD network testing as a service that automates topology setup, resource checkout, retries and health checks and runs 500K+ tests per day for 11 teams across 68 use-cases. It manages roughly O(1k) test devices, finds pathological bugs that evade statistical sampling, and records 50,000 failed tests (140 days) that stop bugs before production. A centralized lab-as-a-service and heavy engineering uncover microsecond race conditions in drain logic and enable biweekly global switch software updates.
AI-written summary · about
chapters (8)
- 0:00Netcastle introduction
- 3:12Why network testing matters
- 6:42Exploding test matrix
- 9:48Network testing as a service
- 11:12Netcastle status and scale
- 12:37Critical bug example
- 14:21Key project lessons
- 17:07Conclusion
related talks
25mNetwork Automation Tools & Technology
37mZero-Regression Network CI/CD for Finance-Grade Reliability
36mNetwork Machinery: A United-Front for Network Troubleshooting with CRDs
56mA Practical Look at NetDevOps
01h 21mIntro + Deep Dive: Cloud Native Network Function (CNF) Testbed
18mNetKeeper: Enhancing Network Resilience with Autonomous Network Configuration...
19mNetEdit: Fine-grained Network Tuning at Scale
45mHighlighting Network Automation Bright Spots with NRE Labs
