Security Logging Use Cases: Building an Open-Source SIEM
summary
Builds a practical open‑source SIEM around Fluentd and Fluent Bit and OpenSearch, showing real plumbing: Kafka for buffering, stream subscriptions instead of polling APIs, and enrichment via eBPF and lookup lists. Notes market scale ($3B) and common operational numbers — ~200 customers with SIEM use, ~90 days of retention is typical — and recommends edge filtering and one percent sampling for successful auths, while keeping all failures, because centralized rule‑based correlation doesn’t scale.
AI-written summary · about
chapters (17)
- 0:00Intro
- 2:29Security Information and Event Management (SIEM)
- 3:54SIEM Use Cases and Users
- 6:00Market Dynamics
- 6:51Open Source Options
- 8:13Data Types
- 9:57Data Sources
- 11:13Data collection methods
- 12:40Data Parsing and Schemas
- 14:51Enrichment Use Cases
- 19:04Ingestion
- 20:38Storage
- 22:39Why Correlation?
- 24:24Rule Based Correlation
- 25:57Incident identification
- 27:09Incident Workflow
- 29:03How to build your own SIEM
related talks
01h 05mPractical network defense at scale
31mFluent Bit & OpenSearch: The Power Couple of Open Source Observability
02mThe Challenges of Legacy SIEM Models
40mSIEM as Needed: Elasticsearch, Energy Logserver, Wazuh...
20mSIEM with Elastic – Insights for Security, Freedom for DevOp
13mSecuring the Data Frontier: Real-World Strategies With OpenSearch
18mFrom Logs To Signals: Rethinking System Monitoring With OpenSearch
46mOperational Logs and Event Management for HPC S...