Imposing a prior on the coefficients of time series regression

Interfaces brms
1

The basic regression model for the time series data \boldsymbol{y}=(y_1,y_2,...,y_n)^T is

\boldsymbol{y} \sim~ N(X\boldsymbol{b}, \boldsymbol{\Sigma})

with the design matrix X=(\boldsymbol{1}, \boldsymbol{x}_1, ..., \boldsymbol{x}_k), the corresponding regression coefficients \boldsymbol{b}=(b_0, b_1, ..., b_k)^T and the residual variance-variance matrix \Sigma. The k predictors, vectors \boldsymbol{x}_1, ..., \boldsymbol{x}_k, have the same length of n as the response variable \boldsymbol{y}.

Can the model be implemented in brms if I want to impose a prior distribution on the regression coefficients (except b_0): b_i \sim N(b, \tau^2) (i=1,2,...,k)? Also, how to specify the autoregressive structure of the residuals with a band matrix \Sigma?

2

Although my understanding of time series is limited, the covariance structure you want looks very similar to the one provided by cor_ar or is there something I am missing? Other than that it should be simple.

3

Yes, the variance-covariance structure for the residuals can be modeled through cor_ar or cor_arma in brms. But how to impose the constraint b_i \sim N(b, \tau^2) on the regressors coefficients?

Thanks!

4

Oh, I see - you want there to be a single \tau shared across all b_i? Then you will probably be best served by horseshoe: https://rdrr.io/cran/brms/man/horseshoe.html, although this is slightly different than your model, it should IMHO serve the same purpose. Or am I missing something?

5

The horseshoe is indeed an interesting idea. However, the horseshoe prior would not work in my case because I just realized there are a couple of issues I didn’t state correctly in my original post.

(1) Actually it is not that I want to impose a prior on some of the population effects, but rather I’d like to assume that b_i follows a Gaussian distribution N(b, \tau^2), where both b and \tau^2 are unknown.

(2) Sometimes the Gaussian assumption is placed only on some, not all, of the population effects b_i.

Is this beyond the capability of brms?

6

Ad 1) That is very much like horseshoe, with two exceptions:

a. The prior in horseshoe is a bit more complicated than normal
b. The prior in horseshoe is centered at zero

Now not centering the effects at zero is weird, but it shouldn’t be much of an issues, because what you wrote is IMHO equivalent to

\hat{b}_i \sim N(0, \tau^2) \\ b_i = b + \hat{b}_i

and then

\mu_i = \sum_{j=0}^k x_{i,j} b_j = \sum_{j=0}^k x_{i,j} \hat{b}_j + b \sum_{j=0}^k x_{i,j}

So you can easily get away with zero-centered effects as long as you add a new fixed effect for sum of the predictors (please double check my math though)

So it is up to whether you are willing to use a slightly different shape than normal for the prior on \hat{b}_i.

You should IMHO be able to handle 2) in this framework as well.

But you are getting close to the limits of brms :-)

1 Like
7

You’re forcing me to have some serious reading now. Thanks a lot!