Achieve real-time inference at scale on AWS with Deep Java Library
summary
Deep Java Library (DJL) loads pretrained models from its model zoo (over 70+ models), using JNI-backed engines like PyTorch and ONNX Runtime so Java inference matches Python C++ performance. The demo runs a PyTorch sentiment model (DistilBERT vocab file shown), embeds a predictor per thread pattern because predictors are not thread-safe, and deploys a Flink job packaged as an S3 JAR to Amazon EMR. Real customers report up to 3x speedup on Spark batch inference versus Python server approaches.
AI-written summary · about
chapters (7)
- 0:00Introduction to DJL
- 1:38Benefits and Architecture
- 5:38Model Zoo exploration
- 9:32Prediction and translators
- 16:41Apache Flink integration
- 20:36Deployment on Amazon EMR
- 24:16Development roadmap and wrap
related talks
01h 01mDeep Learning and Java
25mDeeplearning4j: How to run AI workloads on Jakarta EE compliant servers
51mIntegrate Machine Learning into Your Spring Application in Less than an Hour
02h 37mDeveloping Cloud-Native Java AI applications with DJL and LangChain4J
22mBringing Deep Learning into the Big Data Analytics
44mDeep Learning in Production
04mYuhao Yang Interview
50mML in Java, YES it's possible!