Regularized horseshoe prior for non-GLM models

Modeling
,
1

I am trying to fit a Markov model using Stan (as opposed to brms) with several covariates mediating the rate of transitions between some states. I would be grateful to receive advice on the use of regularized horseshoe prior in this context.

From my reading of “Sparsity information and regularization in the horseshoe and other shrinkage priors” by Juho Piironen and Aki Vehtari (@avehtari), it appears that it is not straightforward to rely on the Eq 3.12 (or its counterparts for other glm families) to fit other models (in my case a Markov model that samples using categorical()).

The recommendation given in Section 3.6 is to “draw from the prior for different values of tau”. Here I am confused between the following options: 1) to try out different fixed values of tau or 2) to sample tau from a Cauchy distribution and try out different scale values.

More generally, I am also wondering if it is reasonable to consider p0 (i.e., the prior guess for nonzero) in Eq 3.12 as a rough guide for choosing tau values in non-GLM settings.

2

Both are valid and produce a different prior for the coefficients

It is a useful approximation as discussed in section 3.5

This is an interesting use case! Our paper discusses the prior specifically from the sparsity point of view, but you might look also at R2D2 type prior which can be used to set prior on proportion of explained variance, although it’s not clear how you would use it for a categorical target.

1 Like