Hey there,

I recently posted for the first time in this forum, and I had such quick and great responses that I can’t help it but ask another question regarding Gaussian processes that has been bugging me for a while (and for which I can’t seem to find anything relevant online). I was hoping someone could shed some light on it or point me towards the right direction.

Imagine that I have two variables \alpha_i and \beta_i for which I have basic information on the differences between indices i. In particular, I want to assume that for parameters \alpha_i those differences are characterised by a distance matrix D^{\alpha}, and for parameters \beta_i the same are characterised by a distance matrix D^{\beta}. As a starting point, I can write those such that their priors are independent multivariate distributions such that

\alpha_i \sim \text{MVNormal}\left(\bar{\alpha}, K^{\alpha}\right),

\beta_i \sim \text{MVNormal}\left(\bar{\beta}, K^{\beta}\right),

\bar{\alpha}, \bar{\beta}\sim \text{Normal}\left(0,1\right),

K^{\alpha}_{ij} = \eta_{\alpha}\,\exp(- \rho_{\alpha} {D^{\alpha}_{ij}}^2) +\delta_{ij}\,\sigma_{\alpha},

K^{\beta}_{ij} = \eta_{\beta}\,\exp(- \rho_{\beta} {D^{\beta}_{ij}}^2) +\delta_{ij}\,\sigma_{\beta},

\sigma_{\alpha}, \sigma_{\beta}, \eta_{\alpha}, \eta_{\beta} \sim \text{Exponential}\left(1\right)

\rho_{\alpha}, \rho_{\beta}\sim \text{Exponential}\left(0.5\right)

Assume, however, that I am expecting some correlation between \alpha_i and \beta_i. I can certainly do a post hoc analysis of this correlation by comparing their posterior distributions; but, I wonder if there is a way to include this correlation in the model, directly. Discussing things with people that know much more than me, I have been suggested multiple ways to do this. One way is by modelling \alpha_i and \beta_i together such that:

\begin{bmatrix}
\alpha_i \\
\beta_i
\end{bmatrix} \sim \text{MVNormal}\left( \begin{bmatrix}
\bar{\alpha} \\
\bar{\beta}
\end{bmatrix}, K\right),

K=\begin{bmatrix}
K^{\alpha} & \nu\\
\nu & K^{\beta}
\end{bmatrix},

where \nu is the correlation between \alpha_i and \beta_i. Unfortunately, I have found little success doing this with simulated data. This could well be a result of me messing up things along the process, but I have a feeling that my covariance structure K is missing something. More specifically, I believe that there should be a factor multiplying \nu that accounts for the covariance between alphas, and the covariance between betas (i.e. K^{\alpha} and K^{\beta}).

Alternatively, I have been suggested to account for this correlation in the prior distribution of the corresponding hyperparameters (i.e. \rho, \sigma and \eta). That is, writing their prior distributions as a multivariate distribution with several \nu measuring the pairwise correlation between hyperparameters. However, I don’t necessarily see how I can then understand the overall correlation between \alpha_i and \beta_i via the correlation of their hyperparameters. Other interesting suggestions that I was given included modelling \alpha_i and \beta_i as correlated via the LKJ and then impose spatial autocorrelation on the residuals following a Gaussian process that include the distance matrices D, but I haven’t had a chance to try this yet.

Not sure if any of this makes sense or if I am missing something trivial or already discussed in the forum/documentation, but it would be wonderful to hear some ideas on how to approach this.

Thanks in advance for your time!