Analyzing Massive Time Series Data with Spark
summary
Open-source Spark-TS library built on Apache Spark and exposing Scala/Java/Python APIs implements a time series RDD that stores each key’s full series plus a datetime index so thousands or millions of independent series stay partition-local. It provides subslice, linear interpolation for missing values, automated AR/ARIMA models (auto-order selection), GARCH, Durbin–Watson and per-series model fitting without cluster shuffles. Future work targets cross-machine sharding of single oversized series and multivariate VAR-style models.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker intro
- 1:59What is time series data
- 3:25Why time series is unique
- 6:12Uses of time series data
- 9:02Existing time series tools
- 10:28Time Series for Spark project
- 11:33Time series data layouts
- 14:19Distributing time series data
- 16:03Library usage demo
- 17:28Future library improvements
- 19:13Q&A on time dependencies
related talks
07mTime Series Databases
38mReal-time analytics with Apache Cassandra & Apache Spark
01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live coding
55mSearching for Time Series Patterns at Big Data Scale
52mSolving Analytical Problems using Apache Spark
01h 36mSpark for Scala Developers
50mIntroduction to Real-Time Big Data with Apache Spark
11mScaling R with Spark