Multivariate horseshoe prior?


In the course of applying regularized horseshoe priors to models in which two output variables respond to the same set of input variables (a la Piironen & Vehtari 2017), I noticed that the coefficients of two different output variables (number of wildfire events and expected wildfire size) are positively related:


I am currently using separate regularized horseshoe priors for these two outputs, but I’ve been considering explicitly modeling a correlation in coefficients among responses by generalizing the univariate horseshoe prior to a multivariate horseshoe prior. This would be consistent with our intuition for this system (conditions that increase the probability of fire occurrence are expected to also increase the size of a fire), and it seems like it might be useful to allow information sharing among the M output dimensions.

So, I wanted to bounce this idea off the group and see if there are any obvious pitfalls that I’m overlooking before diving too deep. Currently, what I’m doing for my M = 2 output variables is something to the effect of:


for each output dimension (m = 1, 2) and each input variable (j = 1, …, D), where lambda could be \tilde{\lambda} for a regularized horseshoe prior.

An equivalent specification places an uncorrelated multivariate normal prior on each length M vector of coefficients for each input j:


for each input variable j =1, …, D.

Written this way, one might be tempted to introduce a parameter rho that allows correlation among the coefficients:


Does this seem reasonable, or am I missing something obviously wrong with this approach?

Edit: maybe you could also share information among the M output dimensions by placing a multivariate Cauchy prior on the lambdas…

Translation to Stan code for a multivariate regularized horseshoe if anyone is interested. mv-hs-ncp.stan (1.4 KB)

We used a similar multivariate extension of horseshoe prior in

Peltola, Havulinna, Salomaa, Vehtari. Hierarchical Bayesian Survival Analysis and Projective Covariate Selection in Cardiovascular Event Risk Prediction

(Our code is also available, but hasn’t been updated to the latest Stan versions:

You could use some predictive comparisons to see if the multivariate approach is useful compared to separate priors.


Thanks for posting up - I like that this specification is able to share information on relevance AND sign/effect size.