Hi there!
I just wanted to share with you a humble trick I just discovered.
I love exponential priors for standard deviation, either residual or of hierarchical effects. I find scaling very intuitive, and it limits divergences in general. I was wondering how to make it un-centered, and if it would improve sampling (as I find an un-centered lognormal prior quite hard to scale, for example).
I found the solution in the wikipedia page of the exponential distribution
The idea is to give a raw parameter a uniform distribution between zero and one, and then to transform it with the formula given in the link above.
data{
real lam_sigma; // Exponential decay rate parameter
}
parameters{
real<lower=0,upper=1> sigma_raw;
}
transformed parameters{
real<lower=0> sigma = -log(sigma_raw)/lam_sigma;
}
In my current application (a multi-level SEM with latent variables), it worked quite successfully at reducing divergent transitions and improving sampling.
Hope that helps!
Have a great day,
Lucas
5 Likes
Nice nice! Good trick. I think this is the trick to sampling a Cauchy as well (sample uniform [0, 1] and then the inverse CDF is easy to compute). There’s a section in the manual with stuff like this: 22.7 Reparameterization | Stan User’s Guide . If you feel inspired to write a little blurb that could be useful.
What was this parameter in your model? Was it a hierarchical variance? Or something else?
2 Likes
Hi!
Yes, I used it for the hierarchical variance of factor scores (latent variables of size N \times D, N being the number of data points and D the number of latent dimensions.
It would be quite exciting to contribute to the manual, I’ll check to write something!
Cheers,
Lucas
Docs are here: GitHub - stan-dev/docs: Online documentation for Stan language and platform
You modify things in src
and then run the script python build.py 2 5000
to build development docs in the folder docs/2_5000
so you can check they build right. I use a weird version so it doesn’t overlap with anything there.
You only commit the source files. The stuff is docs is built by some sort of automation thing I think. For instance, here is an example pr: Add docs for the skew double exponential by dirmeier · Pull Request #336 · stan-dev/docs · GitHub
There is two parts dealing with reparametrization : 21.2 and 22.7. Where do you think it should go?
Thanks!
Lucas
I think just put it with the Cauchy reparameterization.
Do you also have time to try to make a simple model that benefits from this? Like, a toy problem model where it’s clear that changing to the inverse CDF style exponential is the way to go.