Deep Learning in Production
summary
Walkthrough of a JVM-first deep learning stack that trains on GPUs and serves models as Java microservices: Deeplearning4j with ND4J/LibND4J (links to cuDNN and Intel MKL), DataVec for ETL, JavaCPP-generated JNI, and model import from Keras/Caffe. Integrations include mesos + Spark and Kafka; GPUs typically have 12–24 GB RAM so training ImageNet from scratch takes >1 week on one node. Demonstrated features: async GPU memory management, real-time training UI, and TensorRT inference giving ~3–5x production speedups; NASA JPL migrated from TensorFlow to this stack.
AI-written summary · about
chapters (25)
- 0:00Intro
- 2:24Defining Production
- 6:12Research Clusters
- 8:07HPC Video Transcoding
- 11:45Big Data Hybrid Cluster
- 12:55Mesos
- 14:00Spark
- 14:55Inference
- 16:25TensorRT
- 17:12Hiding
- 18:03Latency Hiding
- 19:14Prefetch Buffer
- 19:43Flux
- 20:59Latency
- 22:57Deep Learning Ecosystem
- 24:03Tensors
- 29:12James Weaver
- 30:54GPU Cluster Memory Management
- 32:31GPU Memory Management
- 34:58Kaggle
- 35:24Data Developers
- 35:59NASA
- 37:28Low Latency
- 38:56Pretrained Models
- 40:38System Information
related talks
50mApplying Machine Learning to IoT Use Cases with Deeplearning4j
01h 01mDeep Learning and Java
25mDeeplearning4j: How to run AI workloads on Jakarta EE compliant servers
44mDeep Learning with DeepLearning4J and Spring Boot
40mScaling From Research to Production with Skymind DL4J and ScalNet
22mBringing Deep Learning into the Big Data Analytics
42mDeep Learning is Like Water (it's Everywhere!)
49mDistributed Deep Learning