Understanding Regularized HS prior


#1

I am struggling to understand the various arguments of the hs() prior in rstanarm. I read through Piironen, Vehtari 2017 but I’m still struggling with some things. The calculation of tau0, or the global intercept, makes sense. But I’m having trouble with the others hyperparameters, though I find adjusting them does affect my predictive accuracy. I find the term “slab” particularly confusing. What would be nice is a sense of how the various hyperparameters affects the model. For example, one argument is simply ‘df’. I am unsure what exactly this refers to as there is also a global_df and a slab_df IIRC. If I do a prior_summary call to my model in rstanarm, this df appears as the prior to coefficients. I’m working on a n<<p problem and finding many of my coefficients have very heavily skewed distributions, though I don’t seem to be having any sampling issues. It makes me suspicious some of my priors are reigning the estimates in too much but perhaps I’m misdiagnosising the issue.


#2

hs refers to hierarchical shrinkage, and to get regularized horseshoe set df=1, global_df=1. If you want sparsifying prior, it’s best to leave them like that. Then you need to choose global_scale, slab_df, and slab_scale.

That is the global_scale and it seems you have that figured out.

slab describes the prior for the large coefficients. slab is t-distribution and has scale and df, which you can choose based on your prior information about the magnitude of large coefficients. df is local df or local nu. slab is important, for example, in logistic regression with separable classes as without regularization horseshoe would have way too much mass for unfeasible large weights.

and which likelihood?

posterior distributions? It’s fine if the posterior is skewed.