Streaming Graphs, Because We Can't Afford to Query Any More
summary
Claims a streaming graph called Quine that pushes query execution inline with the event stream and can scale to millions of events per second. Built on Akka using an actor-per-node model, edge‑cut partitioning, an ID‑from hashing primitive, and event‑sourced persistence (Cassandra or RocksDB backends) so nodes can be replayed instead of time‑windowed in RAM. Uses Cipher standing queries that propagate through nodes and in the demo finds complex cybersecurity patterns and publishes results to Kafka in real time.
AI-written summary · about
chapters (5)
- 0:00Start
- 1:10The Streaming Data-Join Problem assemble events into meaningful objects
- 6:09Introducing Quine a streaming graph interpreter for complex event processing
- 13:38Fine-Grained Graph Versioning
- 15:51Standing Query
related talks
27mStream, Store, Visualize: Fast-Track Real-Time Analytics the OSS Way
49mStreaming Visualisation
41mFast Data - an Overview of the Concepts, Architecture and Technology of Streaming Analytics
36mKafka Streaming, the Pythonic Way
45mUsing time-series data to better understand our products & infrastructure
36mData pipelines engineering made simple with Scala
56mReal Time Data Pipelines
07mTransforming Big Data into Real-Time Product Insights