Greetings everyone,
From what I have been reading, we should avoid the use of variational inference (VI) for purposes of inferences as the results can be quite different compared to HMC-NUTS. One question I have is whether the differences between the results of VI and HMC wash out in a simulation study over hundreds/thousands of replications.
Thank you,
David
I think it would depend entirely on the model it’s applied to, and what the simulation is assessing.
VI already doesn’t work well for many models, in my experience. But even for models where the mean estimates are similar to NUTS’, the variances are systematically off for me (usually VI has too little posterior variance compared to NUTS, so it is overly certain).
Personally, I wouldn’t trust VI in simulations without understanding how VI tends to systematically vary from to NUTS for the model and simulation goals at hand; but by the time you do that, you could have already just run the simulation with NUTS anyway.