Excluding intercept while using horseshoe prior in nonlinear model

Interfaces brms
1

Hi
I want to use the horsehoe prior with a nonlinear model, but I suspect that it also regularizing the intercept parameter when it shouldn’t.

Can somebody check if I’m doing something wrong or if it is an issue with brms?

I demonstrate the issue with 2 toy data sets with the only difference that there is a bias in the second dat set translating in a large intercept value.

library(brms)
set.seed(123)
Y = sample(c(0,1),100,replace= TRUE)
a = rnorm(Y,Y)
b = rnorm(Y,Y)
c = rnorm(Y,Y)
d = rnorm(Y,Y)
e = rnorm(Y,Y)
f = rnorm(Y,Y)
dd1 = data.frame(Y,a,b,c,d,e,f)
## same data with added bias
dd2 = dd1
dd2[-1] = dd2[-1] + 10

Now I fit a linear brms model with horseshoe prior

fit1 = brm(Y~., family = bernoulli("logit"), data = dd1,prior = prior(horseshoe())
        , seed = 1,eta = 1,adapt_engaged = FALSE, init = 0, algorithm = 'meanfield')
fit2 = brm(Y~., family = bernoulli("logit"), data = dd2,prior = prior(horseshoe())
         , seed = 1,eta = 1,adapt_engaged = FALSE, init = 0, algorithm = 'meanfield')

cbind(fixef(fit1)[,1],fixef(fit2)[,1])

Indeed, all parameter estimates are the same except the intercept

Intercept -4.1621522 -84.3192850
a          1.0886389   1.0886389
b          1.3354069   1.3354069
c          1.5518533   1.5518533
d          1.4961902   1.4961902
e          0.8681363   0.8681363
f          1.6754869   1.6754869

Now, I fit the same data sets with a nonlinear model

form <- bf(Y ~ log(inv_logit(eta)), eta ~ a + b + c + d + e + f, nl = TRUE)
fit3 = brm(form, family = bernoulli("logit"), data = dd1,prior = set_prior(horseshoe(), nlpar = 'eta')
        , seed = 1,eta = 1,adapt_engaged = FALSE, init = 0, algorithm = 'meanfield')
fit4 = brm(form, family = bernoulli("logit"), data = dd2,prior = set_prior(horseshoe(), nlpar = 'eta')
         , seed = 1,eta = 1,adapt_engaged = FALSE, init = 0, algorithm = 'meanfield')

cbind(fixef(fit3)[,1],fixef(fit4)[,1])

All parameters are different.

eta_Intercept -1.9768419 0.03578799
eta_a          0.8661051 0.39018095
eta_b          2.2615091 0.42025410
eta_c          1.7398894 0.66441369
eta_d          2.0722380 0.31776399
eta_e          1.2938160 0.23838384
eta_f          2.0037543 0.19358062

To me, it looks like if the intercept is also shrunk by the horseshoe prior.
How can I prevent that? Am I specifying the model correctly?

Thanks in advance

  • Operating System: Linux
  • brms Version: 2.7
2

Yes, in non-linear models the intercept is treated as part of the vector of regression coefficients and thus is also subject to shrinkage via the horseshoe prior. I agree this is unfortunate. Let me see if I can find a good solution to turn this off and treat the intercept as separate just as is the case for linear models.

3

You can now control this behavior via argument center of bf and related functions in the github version of brms. For your example, try out

form <- bf(Y ~ log(inv_logit(eta)), nl = TRUE) + 
  lf(eta ~ a + b + c + d + e + f, center = TRUE)
2 Likes
4

Works like a charm!
Thanks!