For didactic purposes, I’d like to implement a simple hierarchical (logistic) regression model with Normal heterogeneity and LKJ/Cauchy priors on correlation/variance in plain R, i.e. coding it up from scratch.
I’d want to use different samplers, including a rather basic Metropolis-Hastings, simple HMC, and maybe also NUTS. They key challenge I see is to generate reasonable proposals. The lkj distribution does not have a location matrix parameter, so it’s difficult to achieve a reasonable acceptance rate. A Wishart distribution is better in that regard, but it’s proposal would need to be transformed to a correlation matrix, and I am not sure about the ratio of proposal densities there. The same is true for transformed Normal randomness on the elements of the Cholesky.
I think stan uses some smart transformations to do this, but I wasn’t able to find any documentation. Can someone point me to useful resources?
Stan samples everything in an unconstrained space, which makes things like proposals simpler.
The docs for these transformations are here: https://mc-stan.org/docs/2_22/reference-manual/variable-transforms-chapter.html
During adaptation, by default Stan tries to figure out the marginal variance in the direction of each unconstrained variable and then rescale its coordinates by that to make it sample faster. That process is described here: https://mc-stan.org/docs/2_22/reference-manual/hmc-algorithm-parameters.html
Either of those things what you were looking for?
3 Likes
Yes, thank you. I don’t know how I missed this in the manual.
1 Like
There’s a lot going on haha. Easy to miss stuff.
1 Like