When applying the ADVI full rank method for bayesian analysis, are the posteriors of the parameters distributed as normal? Is the mean and sd reported using the summary on my fitted model object the posterior means and standard deviations?
On a similar topic, if I were to select the full rank option for the variational parameters, how could I obtain the off diagonal elements of the covariance matrix?
Oh I got my answer going through the documents and reading the Kucukelbir paper thoroughly. Sorry!
Just to help everyone else out, what’s happening is that the variational approximation is multivariate normal on the unconstrained scale. So that means if you have something like a positive-constrained paameter, it will have a lognormal distribution on the constrained scale. Similarly for other parameters.
The person to follow on this is Tamara Broderick—she and her group at MIT are doing a lot of work around estimating covariance in approximate variational models.