Uncentered exponential priors and hyperpriors

Modeling
,
1

Hi there!

I just wanted to share with you a humble trick I just discovered.

I love exponential priors for standard deviation, either residual or of hierarchical effects. I find scaling very intuitive, and it limits divergences in general. I was wondering how to make it un-centered, and if it would improve sampling (as I find an un-centered lognormal prior quite hard to scale, for example).

I found the solution in the wikipedia page of the exponential distribution

The idea is to give a raw parameter a uniform distribution between zero and one, and then to transform it with the formula given in the link above.

data{
real lam_sigma; // Exponential decay rate parameter
}
parameters{
real<lower=0,upper=1> sigma_raw;
}
transformed parameters{
real<lower=0> sigma = -log(sigma_raw)/lam_sigma;
}

In my current application (a multi-level SEM with latent variables), it worked quite successfully at reducing divergent transitions and improving sampling.

Hope that helps!
Have a great day,
Lucas

5 Likes
2

Nice nice! Good trick. I think this is the trick to sampling a Cauchy as well (sample uniform [0, 1] and then the inverse CDF is easy to compute). There’s a section in the manual with stuff like this: 22.7 Reparameterization | Stan User’s Guide . If you feel inspired to write a little blurb that could be useful.

What was this parameter in your model? Was it a hierarchical variance? Or something else?

2 Likes
3

Hi!

Yes, I used it for the hierarchical variance of factor scores (latent variables of size N \times D, N being the number of data points and D the number of latent dimensions.

It would be quite exciting to contribute to the manual, I’ll check to write something!

Cheers,
Lucas

4

Docs are here: GitHub - stan-dev/docs: Online documentation for Stan language and platform

You modify things in src and then run the script python build.py 2 5000 to build development docs in the folder docs/2_5000 so you can check they build right. I use a weird version so it doesn’t overlap with anything there.

You only commit the source files. The stuff is docs is built by some sort of automation thing I think. For instance, here is an example pr: Add docs for the skew double exponential by dirmeier · Pull Request #336 · stan-dev/docs · GitHub

5

Excellent, thank you!

Lucas

6

There is two parts dealing with reparametrization : 21.2 and 22.7. Where do you think it should go?

Thanks!
Lucas

7

I think just put it with the Cauchy reparameterization.

Do you also have time to try to make a simple model that benefits from this? Like, a toy problem model where it’s clear that changing to the inverse CDF style exponential is the way to go.

8

Gentlemen, I tried implementing this on my hierarchical model. I also like to use exponential hyperpriors for random effect standard deviations. Interestingly, the sampler run the warmup iterations and then aborted the sampling iterations, claiming “Assertion failed.” Only time I’ve ever seen that happen. Any idea what’s going on there?