JVM based DeepLearing on IoT data with Apache Spark
summary
Shows a runnable pipeline that trains an LSTM autoencoder on IoT sensor streams using DeepLearning4J on Apache Spark, running distributed on CPU/GPU clusters and scaling out via Spark executors. Describes how ApacheSystemML uses cost‑based optimizers to generate execution plans and how TensorSpark parallelizes TensorFlow on Spark, and demonstrates deployed uses in IBM Watson Developer Cloud for Visual Recognition, psychological profiling and document classification. Includes a working anomaly detector example and deployment notes.
AI-written summary · about
chapters (10)
- 0:00Introduction to talk
- 2:36Machine learning basics
- 8:08Linear regression explained
- 11:09Spark for data processing
- 15:22DataFrames and SQL in Spark
- 19:50Apache Spark in IBM Watson
- 24:41High-level operations in Spark
- 29:02Demo: Physical model with Spark
- 33:26Neural network training demo
- 34:40Q&A and closing
related talks
50mApplying Machine Learning to IoT Use Cases with Deeplearning4j
54mDeep Learning with Apache Spark
22mBringing Deep Learning into the Big Data Analytics
55mLarge Scale Distributed AI on the JVM
50mDeep Learning and NLP with Spark
45mDeep Learning and NLP with Spark
25mDeeplearning4j: How to run AI workloads on Jakarta EE compliant servers
04mYuhao Yang Interview