Deploying ML models in a Clojure environment
summary
Nubank adopts a Clojure side-car that proxies real-time inference so Python model containers only talk to localhost; this removes duplicated plumbing (auth, logging, config) and standardizes Python servers, supporting about ~100 realtime models in production. Experiments with ONNX runtime load models directly into JVM services, yielding 2–3× faster single-request inference on a 10-feature logistic/regression benchmark and much larger speedups in public tests; migration cost, feature-engineering coupling, and legacy risk block wholesale rollout.
AI-written summary · about
chapters (9)
- 0:00Introduction and context
- 1:30The challenges of polyglot services
- 3:24The AI/ML deployment struggle
- 6:33Initial attempts at solutions
- 9:48The sidecar architecture
- 15:52Interoperability via ONNX
- 23:16Implementing ONNX in practice
- 31:49Performance benchmarking
- 36:55Challenges and future outlook
related talks
30mAccelerating AI Application: NAVER's Journey from Model to Application
40mBuilding Machine Learning Models with Clojure and Cortex
25mMore Clojure Less Complication
33mDealing with the versioning of production-ready models
40mBuilding highload ML powered service
12mPreview: ML Serving
30mScalable ML acceleration with ONNX Runtime
21mEdge Inference on Unknown Models