Event-driven Agents with Complex Event Processing in Flink
summary
Claims a concrete architecture: use Apache Flink CEP to detect sequences and filter millions‑per‑hour Kafka events, run statistical anomaly detection or simple classifiers first, then call LLMs only for the filtered cases to cut model costs and reduce hallucinations. Implements patterns with Flink SQL match_recognize (example: price jump + drop within 1 hour) and a second‑stage LLM for classification/action. Shows real examples (stock ticks, predictive maintenance, ERP process‑mining), argues small models give deterministic outcomes and the pipeline works at scale.
AI-written summary · about
chapters (19)
- 0:00Introduction to event-driven agents
- 1:56Agent sprawl and orchestration challenges
- 3:41Why data consistency matters for AI
- 7:13Apache Flink’s role in streaming agents
- 10:43Event-driven vs analytical agents
- 12:47Cost efficiency in streaming agents
- 14:11Complex Event Processing in Flink
- 16:17Flink SQL for pattern recognition
- 18:04Predictive maintenance use case
- 19:50Event-driven architecture for scalability
- 20:57Flink SQL vs Kafka Streams
- 23:48AI-generated risk patterns
- 25:33Log analysis and pattern generation
- 26:59Monitoring ERP data in real time
- 29:05Kafka as context engine for agents
- 31:32Continuous context updates for agents
- 34:41Governance and compliance for agents
- 36:05Feedback loops for agent improvement
- 38:55Q&A on agent orchestration
related talks
44mFlink Jobs as Agents – Stream Processing for Agentic AI
32mEvent-Driven & Multi-Agent Architectures for Gen AI Workflows
50mHarnessing Event-Driven and Multi-Agent Architectures for Complex Workflows in Gen...
42mAgentic AI with AWS Streaming
24mBuilding Real-Time AI Decision Systems with LLMs
56mHerding LLMs: Predictable Outcomes from Unpredictable Agents
24mReal-Time Context Engineering for LLMs
16mEffective agent design patterns in production