Use of horseshoe prior under complete separation

Modeling
1

I am interested in fitting a logistic model with a horseshoe prior as done in the paper “Sparsity information and regularization in the horseshoe and other shrinkage priors” (https://arxiv.org/abs/1707.01694). I have the following concern, however.

When p > n, there is an issue of complete separation and a posterior will be as heavy-tailed as a prior in some directions. Since a horseshoe prior has a Cauchy-like tail (Carvalho, 2008), it seems to me that posterior means of regression coefficients may not be well-defined.

Let me know if this is an issue and, if not, why that’s the case.

2

The regularized horseshoe introduced in that paper solves the separation problem. The regularized horseshoe changes the shape of the tail. See more in the paper.

2 Likes
3

Thank you. It turns out I was reading another 2017 paper of yours (“On the Hyperprior Choice for the Global Shrinkage Parameter in the Horseshoe Prior”). The regularized horseshoe indeed solves the issue.