Extending Apache Flink stream processing with Apache Samoa ML methods
summary
Integration connects Apache Flink to the stream ML engine Apache SAMOA via Kafka connectors so each labeled instance updates models online and predictions flow back into Flink for further joins. Implemented JSON serializers, a per-instance prediction sink, and fixes to some SAMOA algorithms; demo uses the airline dataset with a vertical Hoeffding tree classifier that evolves after every instance and converges toward about 80% accuracy, with noted network latency and serialization overhead.
AI-written summary · about
chapters (8)
- 0:00Introduction to Flink and Samoa
- 1:06Apache Samoa overview
- 3:13Extending Samoa for production
- 8:15Integration approaches with Flink
- 13:14Demo setup and data flow
- 18:45Source function implementation
- 24:07Production goals and 5G use case
- 26:37Q&A session
related talks
40mStreaming ML with Flink
15mMetamodelization and Automatic Generation of ML Pipelines
16mAnalysing Real time Activity Streams for Security
27mData enrichment patterns with Apache Flink
41mStreams to the River: Powering an Online XGBoost Classifier with Kafka
21mReal-time Stream Analytics and Scoring Using Apache Flink, Druid & Cassandra
42mHow to Leverage the Apache Kafka Ecosystem to...
28mEnabling Machine Learning with Apache Flink