Data-oriented Design for Differentiable, Probabilistic Programming
summary
Reimplementation of the lazy rose tree from LazyPPL into a strict, data-oriented Rust backend reduces thunk size from about 16 to 4 bytes, stores nodes and thunk states in separate vectors, and uses array-of-structs→struct-of-arrays layout. Benchmarks report a Haskell run taking ~5 minutes vs Rust ~2 seconds on piecewise linear MCMC; memory becomes constant across runs by reusing the tape. A reverse-mode autodiff graph for gradients is integrated, and some unsafe Rust is used to implement splitable RNG state safely.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 0:23Motivation
- 1:07Non-parametric statistics
- 2:53Lazy vs classic PPLs
- 3:57Lazy PPL with rose trees
- 5:25Rose tree structure
- 6:08Lazy data in Haskell
- 7:15Data-oriented design
- 7:56Trees to vectors
- 8:39Efficient thunks in Rust
- 9:25AoS to SoA optimization
- 9:47Ergonomic Rust interface
- 10:50Performance benchmarks
- 11:53Beyond speed
- 12:35Future research
- 13:16Q&A
- 13:38Memory reduction
- 15:49Rust's ownership and mutable resource splitting
- 17:44Laziness in Rust DSL
- 19:30Closing
related talks
15mScaling Optimization Over Uncertainty via Compilation
19mIncremental Computation for Efficient Programmable Inference in Probabilistic Programs
29mProgramming machine learning algorithms in hardware, sanely, using Haskell and Rust!
22mStochastic Lazy Knowledge Compilation for Inference in Discrete Probabilistic Programs
35mScalable structure learning and inference for domain-specific probabilistic prog
32mProbabilistic and Differentiable Programming in Scientific Simulators
10mWriting the Fastest GBDT Library in Rust
47mThe State of the Art for Probabilistic Programming