Multivariate horseshoe prior?

Stanimals,

In the course of applying regularized horseshoe priors to models in which two output variables respond to the same set of input variables (a la Piironen & Vehtari 2017), I noticed that the coefficients of two different output variables (number of wildfire events and expected wildfire size) are positively related:

Rplot

I am currently using separate regularized horseshoe priors for these two outputs, but I’ve been considering explicitly modeling a correlation in coefficients among responses by generalizing the univariate horseshoe prior to a multivariate horseshoe prior. This would be consistent with our intuition for this system (conditions that increase the probability of fire occurrence are expected to also increase the size of a fire), and it seems like it might be useful to allow information sharing among the M output dimensions.

So, I wanted to bounce this idea off the group and see if there are any obvious pitfalls that I’m overlooking before diving too deep. Currently, what I’m doing for my M = 2 output variables is something to the effect of:

image

for each output dimension (m = 1, 2) and each input variable (j = 1, …, D), where lambda could be \tilde{\lambda} for a regularized horseshoe prior.

An equivalent specification places an uncorrelated multivariate normal prior on each length M vector of coefficients for each input j:

image

for each input variable j =1, …, D.

Written this way, one might be tempted to introduce a parameter rho that allows correlation among the coefficients:

image

Does this seem reasonable, or am I missing something obviously wrong with this approach?

Edit: maybe you could also share information among the M output dimensions by placing a multivariate Cauchy prior on the lambdas…

Translation to Stan code for a multivariate regularized horseshoe if anyone is interested. mv-hs-ncp.stan (1.4 KB)

We used a similar multivariate extension of horseshoe prior in

Peltola, Havulinna, Salomaa, Vehtari. Hierarchical Bayesian Survival Analysis and Projective Covariate Selection in Cardiovascular Event Risk Prediction
http://ceur-ws.org/Vol-1218/bmaw2014_paper_8.pdf

(Our code is also available, but hasn’t been updated to the latest Stan versions: https://github.com/to-mi/stan-survival-shrinkage)

You could use some predictive comparisons to see if the multivariate approach is useful compared to separate priors.

2 Likes

Thanks for posting up - I like that this specification is able to share information on relevance AND sign/effect size.