Bringing Deep Learning into the Big Data Analytics
summary
A distributed deep learning library on Apache Spark implemented in Scala with a Python API and native C bundled as a single jar uses Intel MKL to accelerate tensor ops and exposes over 200 layers and models like VGG, SSD and Faster R‑CNN. Production use includes a fraud system with extreme imbalance (~2 positives per 10,000) using ensemble neural nets and resampling, a recommender with 25k‑dim embeddings, and a claim of CPU parity with GPUs (often ~80% and in one case 3x faster) while loading TensorFlow/Caffe models.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker background
- 1:51BigDL core components overview
- 3:40Implemented deep learning models
- 4:47Deployment and ecosystem support
- 6:20Industry use cases introduction
- 9:11MLS Listing image similarity
- 13:30GigaSpace speech recognition
- 14:34UnionPay fraud detection
- 17:07China Life recommender system
- 18:58Function parity and performance comparison
- 20:50Scala vs Python interface usage
related talks
54mDeep Learning with Apache Spark
42mScalable Data Science in Python and R on Apache Spark
40mScala and Spark in the New Deep Learning Era
39mMachine Learning with Scala on Spark
44mDeep Learning in Production
49mDistributed Deep Learning
35mJVM based DeepLearing on IoT data with Apache Spark
21mConvolutional Neural Networks at scale in