Uncentered exponential priors and hyperpriors

Modeling
,
#1

Hi there!

I just wanted to share with you a humble trick I just discovered.

I love exponential priors for standard deviation, either residual or of hierarchical effects. I find scaling very intuitive, and it limits divergences in general. I was wondering how to make it un-centered, and if it would improve sampling (as I find an un-centered lognormal prior quite hard to scale, for example).

I found the solution in the wikipedia page of the exponential distribution

The idea is to give a raw parameter a uniform distribution between zero and one, and then to transform it with the formula given in the link above.

data{
real lam_sigma; // Exponential decay rate parameter
}
parameters{
real<lower=0,upper=1> sigma_raw;
}
transformed parameters{
real<lower=0> sigma = -log(sigma_raw)/lam_sigma;
}

In my current application (a multi-level SEM with latent variables), it worked quite successfully at reducing divergent transitions and improving sampling.

Hope that helps!
Have a great day,
Lucas

5 Likes
#2

Nice nice! Good trick. I think this is the trick to sampling a Cauchy as well (sample uniform [0, 1] and then the inverse CDF is easy to compute). There’s a section in the manual with stuff like this: 22.7 Reparameterization | Stan User’s Guide . If you feel inspired to write a little blurb that could be useful.

What was this parameter in your model? Was it a hierarchical variance? Or something else?

2 Likes
#3

Hi!

Yes, I used it for the hierarchical variance of factor scores (latent variables of size N \times D, N being the number of data points and D the number of latent dimensions.

It would be quite exciting to contribute to the manual, I’ll check to write something!

Cheers,
Lucas

#4

Docs are here: GitHub - stan-dev/docs: Online documentation for Stan language and platform

You modify things in src and then run the script python build.py 2 5000 to build development docs in the folder docs/2_5000 so you can check they build right. I use a weird version so it doesn’t overlap with anything there.

You only commit the source files. The stuff is docs is built by some sort of automation thing I think. For instance, here is an example pr: Add docs for the skew double exponential by dirmeier · Pull Request #336 · stan-dev/docs · GitHub

#5

Excellent, thank you!

Lucas

#6

There is two parts dealing with reparametrization : 21.2 and 22.7. Where do you think it should go?

Thanks!
Lucas

#7

I think just put it with the Cauchy reparameterization.

Do you also have time to try to make a simple model that benefits from this? Like, a toy problem model where it’s clear that changing to the inverse CDF style exponential is the way to go.