Hi everyone,
NeurIPS 2020, a large conference on machine learning, is coming up this week and I thought we could use this thread to share papers, talks, and other events which may, broadly speaking, be relevant to the Stan community.
We can update this thread as we the week unfolds.
I’ll start by sharing the work by @avehtari, Daniel Simpson, Raj Agrawal, and myself on Hamiltonian Monte Carlo using an adjoint-differentiated Laplace approximation [link]. Our poster session is on Thursday at noon (ET), town C2, spot B2. We presented a sister notebook at StanCon 2020 with some demo code on how to use the tools we are developing. Our Neurips article focuses more on the algorithmic details, with a healthy dose on automatic differentiation. Feel free to stop by for a chat!
Pinning @AlejandroCatalina and @andrewgelman who I believe are also presenting.
9 Likes
Akash Dhaka will present Robust, Accurate Stochastic Optimization for Variational Inference which is joint work with @AlejandroCatalina, Michael R. Andersen, @mans_magnusson, Jonathan Huggins, and me. The poster session is Thu Dec 10 09:00 AM – 11:00 AM (PST) @ Poster Session 5 #1477.
The article describes two simple ways how ADVI in Stan could be improved and limitations of ADVI even if the approximating family includes the true posterior. We also illustrate that even if the approximating family includes the true posterior the currently used stochastic optimization makes it likely that in case of moderate and high dimensional posterior the optimization result is far from the target. We can improve by 1) using convergence diagnostics as stopping rule (instead of ELBO threshold) and 2) using average iterations after convergence. These make the optimization more robust and accurate with less variation if re-run. After we know how to make more robust and accurate stochastic optimization we can start considering alternate approximate distributions and divergences etc. The presented stochastic optimization improvements are not yet implemented in Stan C++.
4 Likes
It seems the information has changed and Akash will present in the same poster session as @charlesm93, ie, poster session 6 and if you go that session page it will also show the session time in you local timezone.