In finite dimensional models, the Central Limit Theorem can usually be applied to prove that the posterior distribution converges to a multivariate normal as the amount of data grows. However, Stan can have trouble sampling from multivariate normals when the parameters are highly correlated. My idea is to use draws from an initial “bad” parameterization to locate a better one.
Suppose f(y|theta) is the model and f(theta) is the prior. What do you think about the following general strategy:
- Obtain an initial set of draws from the posterior f(y|theta)f(theta) in the usual fashion
- Use the draws to compute a posterior mean vector mu and a posterior covariance matrix Sigma
- Reparameterize the model as f(y|mu + L_Sigma * epsilon)f(mu + L_Sigma * epsilon), where L_Sigma is the Cholesky factor of Sigma
- Obtain draws from the posterior of epsilon
- Iterate to update mu and Sigma
I’ve had some success with this approach in practice, as the posterior of epsilon is usually “close” to a multivariate standard normal.