I have seen that the regularized horseshoe has been offered as a continuous counterpart of a spike and slab. However, is there any continous counterparts to a spike and slab with a slab that is not centred at zero?
Any discrete spike-and-slab prior can be approximated by a mixture of continuous priors, specifically,
\pi_{\text{spike}}(\beta | \mu_{\text{spike}}) = \delta_{\mu_\text{spike}}(\beta) \approx N(\beta | \mu_{\text{spike}}, \epsilon^2)
for some small \epsilon.
It follows that
\pi_{ss}(\beta) = \gamma \pi_{\text{slab}}(\beta) + (1 - \gamma) \delta_{\mu_\text{spike}}(\beta) \approx \gamma \pi_{\text{slab}}(\beta) + (1 - \gamma) N(\beta | \mu_{\text{spike}}, \epsilon^2).
1 Like
rstanarm and brms have the slab mean fixed at 0, but if you write your model in Stan language, you can use the code example in the Appendix C of the paper Sparsity information and regularization in the horseshoe and other shrinkage priors, and set the slab mean to be non-zero.
1 Like
Hi @avehtari,
Sorry for the late reply, but I think I am not getting it as I dont see where I can set the slab mean from Appendix C. I only see the ability to control the slab scale instead.
You are right it’s not there explicitly. The slab part is explicitly shown in Equation 2.9. The first normal is the scale mixture component of horseshoe and the second normal is the slab. You can change the mean of the slab normal, multiply the two normals, and the result is normal with non-zero mean which is a weighted mean of 0 and the new slab mean (see, e.g. BDA3 eq 2.10). This new mean (say mu) is then added to beta
mu = ...;
beta = mu + z .* lambda_tilde * tau ;
1 Like