Preprint on HMC using an embedded Laplace approximation

@avehtari, @anon75146577, Raj Agrawal and I posted a preprint on the coupling of Hamiltonian Monte Carlo with a Laplace approximation.

Note that several individuals recognized in the acknowledgement section are members of the Stan community and that throughout this project this forum has been an invaluable resource. Indeed this collaboration started with Dan’s post from 2017, initiated with the emphatic sentence “Soooooooooooooooo.” The article constitutes an important milestone in our work on scaling up Bayesian inference for hierarchical models and sheds light on the role the Laplace approximation can play.

Much of our effort was devoted to working out the engineering details. On a test problem, the evaluation and differentiation of a log density with a sensible implementation in Stan took 2,000s, and now runs in 0.1s. We were also able to exploit important principles of automatic differentiation and, in my view, provide a compelling application thereof.

It remains to say that this is a preprint and a milestone in an ongoing project with much work still lying ahead of us.

16 Likes