I wanted to ask in general if people had experience estimating normalizing constants after doing MCMC with Stan. How reliable is bridge sampling and is there a straightforward way to apply it on top of Stan?
One bonus for VI is that, in any methods you mentioned, bridge sampling, path sampling, and SMC, you need some base measurement. The Bridge sampling package uses a version of Laplace approximation for this purpose. If you already have built a VI approximation, it could be used as a better approximation, and thereby result in a more accurate p(y) estimation.
To give some additional context: I’m studying the entropy loss when using VI (think multivariate generalization of variance shrinkage). Much of the analysis assumes the target is Gaussian. I would like to empirically check some results on non-Gaussian targets (e.g. horseshoe models) and ideally get an estimate of the target’s entropy.
I’m concerned about relying on a Gaussian proposal to get an estimate of the entropy, since the whole point is to study the non-Gaussian case. I suppose there is no way around using a good proposals.