PANEL - Streaming.
summary
Panel surveys production streaming: why it matters, how teams trade latency, cost and correctness, and which tools actually ship. Examples include using a stateful stream processor to keep training and inference transformations identical, Apache Pinot for user-facing analytics, Structured Streaming and Apache Pulsar for pipelines, and object-store approaches like WarpStream to cut cost. Concrete patterns: materialized views backed by incremental stream updates, RAG for LLM freshness, connectors, observability and legacy modernization.
AI-written summary · about
chapters (11)
- 0:00Panel introduction
- 1:08Panelist introductions
- 4:42Why streaming is happening
- 11:44Streaming in microservices
- 17:44Challenges with streaming
- 23:19Cost and latency trade-offs
- 31:08Real-time vs business needs
- 36:04Legacy systems and streaming
- 43:09Future streaming tech
- 51:07Streaming in the age of AI
- 1:00:23Audience Q&A
related talks
47mExpert Panel: Current & Future State of the Art in Real-Time Data Infrastructure
45mData Stream Processing Concepts and Implementations
28mSweet Streams Are Made Of These
42mPrinciples and Patterns for Streaming Data Analysis
27mStream, Store, Visualize: Fast-Track Real-Time Analytics the OSS Way
35mStreaming and Real-Time Analytics
41mStreaming: an Alternative to CRUD & Batch Processing
53mWhat it takes for stream data processing