Edge Inference on Unknown Models
summary
Adobe Target enables running customer-provided ML models on Java edge servers by converting models to ONNX runtime and keeping inference inside the existing Java process (no external service), with a model repository and a validation job every two hours that unpacks, checks inputs/outputs, runs a test inference, and enforces p99 serving targets (~10 ms). A small library emits a feature-processing DSL so training and edge pipelines match; JNI native bindings and cross-platform ONNX packaging required extra engineering. Sandbox limits and early-stop support prevent noisy‑neighbor DoS, and the system is already in production use.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:06Adobe Target
- 6:37Solutions
- 14:10Problems
- 14:58Model Validation
- 17:25Model Update
- 17:55Platform Support
- 18:34CrossPlatform Jar Packaging
- 19:13Summary
related talks
36mApplied AI on the edge
27mModel Serving at the Edge Made Easier
30mScalable ML acceleration with ONNX Runtime
37mDeep Learning Neural Network Acceleration at the Edge
26mBuilding a Cloud Native Edge Platform for Accelerating AI Inference
37mDeep Learning Neural Network Acceleration at the Edge
10mThe Rise and Impact of Edge AI
26mBuild your Machine Learning Model on Edge with React Native