Building a Network Telemetry Platform
summary
A concrete platform collects billions of events per day via lightweight agents using TCP dump and Windows dump, ships them into a datastore, and runs a streaming big-data enrichment pipeline that attaches VM metadata and IP‑range mappings. The platform evaluates traffic against explicit baselines (eg allowlist rules for SQL servers), surfaces non‑confirmed flows to alerting, and demonstrates capacity and segmentation recommendations; the implementation scales and supports cloud provider flow logs or agentless ingestion.
AI-written summary · about
chapters (9)
- 0:00intro
- 0:14about netword telemetry
- 1:19the need for network telemetry
- 3:19building the network telemetry platform
- 4:25platform architecture and data processing
- 8:09implementing network security
- 11:46leveraging cloud solutions
- 12:39extending the platform
- 14:01conclusion and final thoughts
related talks
29mFinding the Needle in a Haystack: Tenant-Level Network...
05mTelemetry Pipelines: Your Recipe for Success
45mMastering Distributed Tracing: Insights from Building a Platform from Scratch
05mCollecting and Processing OTEL Data at Scale
49mA Real-World Example of Using Network Behaviour to Detect Cyber-Compromise
01h 05mPractical network defense at scale
27mCollect All the Data; Protect All the Things
08mNetwork Performance Monitoring