Pitfalls of Full Bayesian Inference in Universal Probabilistic Programming
summary
Argues that full Bayes on universal PPLs treats programs as a Bayesian Model Average over their straight-line programs (SLPs), which can make posterior SLP weights collapse to near 0 or 1 even when the true data generating process is not any SLP. Demonstration: a Pyro example with two SLPs estimating a Gaussian mean under different fixed stds, repeated datasets show extreme weight collapse. Proposes stacking—optimizing log posterior predictive density on held-out data—which yields more robust weights and improved predictive performance; work in progress.
AI-written summary · about
related talks
20mStatic Posterior Inference of Bayesian Probabilistic Programming via Polynomial Solving
24mExact Bayesian Inference for Loopy Probabilistic Programs Using Generating Functions
31mProbabilistic Programming: The What, Why and How
01h 03mA Personal Viewpoint on Probabilistic Programming
14mMulti-Language Probabilistic Programming
01h 03mAn Overview of Probabilistic Programming
38mBlack-Box Variational Inference for Probabilistic Programs
01h 07mProbabilistic Programming and Readable Models