FlinkDTW: time-series pattern search at scale using Dynamic Time Warping
summary
An open‑source SIGKDD 2012 C DTW implementation is ported to Java and embedded in Flink to return TopK matches over past or streaming data, with pruning that cuts roughly 95% of DTW computation. On EDF frequency data (166M points) a single thread completes a batch scan in under a minute — about 3M points/sec. Sliding/jumping windows show a 10k-window search in ~20 ms and a 1M-window still runs in milliseconds; streaming requires careful partition/jump handling and is CPU/network intensive.
AI-written summary · about
chapters (10)
- 0:00Introduction to time series
- 2:34Speaker background and context
- 5:53Time series use cases
- 12:52Dynamic time warping
- 17:39Algorithmic optimization
- 22:47Industrial case study
- 26:02Implementation challenges
- 33:20Performance benchmarks
- 35:43Conclusion and future work
- 37:55Q&A session
related talks
55mSearching for Time Series Patterns at Big Data Scale
35mSignatures, patterns and trends: Timeseries data mining @ Etsy
27mData enrichment patterns with Apache Flink
27mDataEngBytes 2023 - MEL-T1-08 - Troy Sellers
41mPredictive Maintenance with Apache Flink
23mData driven matchmaking streaming at HyperConnect
34mWorking with Tensorflow from the JVM #bbuzz
24mTed Dunning, Chief Application Architect, MapR