Bayesian kernel machine regression in Stan

[jroon]

Oh there are sometimes divergences and EFMI warnings. Having multiple r or single doens’t change these degeneracies much I have found.

@huffyhenry

Again, there is no need to estimate hhh if you have a normal observational model. Integrating it out like in the user guide or my code above should make debugging easier.

Ok I have trouble with this whole intergrating out a continuous variable concept. Can you possibly elaborate how your model does this? To me your model is different as you have the GP on the covariance matrix for y, whereas in the bkmr equations h is something added to the mean:

@jroon

Screenshot 2023-06-30 at 20.25.541406×174 13.5 KB

[huffyhenry]

If you have divergences, then you need to resolve them first. There is a lot of advice on this forum and elsewhere how, but I would start by reparametrising like we’ve been discussing.

Starting from the equations that you posted, it should be possible to obtain

\mathbf{y | \beta, \sigma^2, X, \tau, r, Z \sim N(X \beta, \tau K_{Z,r} + \sigma^2I)}

by integrating the first equation over \mathbf{h}. I would appreciate it if someone posted an explicit derivation because I couldn’t make it work, nevertheless it is true: this result appears in Stan user’s guide §10.3 and GPML §2.2. It is also right there at the bottom of the same page of the supplementary info from which you screenshotted the likelihood.

Intuitively, BMKR posits that the (observed) outcome is the sum of the (fixed) effect of covariates \mathbf{X\beta}, the (latent, random) exposure effect \mathbf{h \sim N(0, K)} and the observation error \mathbf{\varepsilon \sim N(0, \sigma^2I)}. But the sum of multivariate normals is multivariate normal with mean vectors and covariance matrices added together, and adding a constant vector just shifts the mean, giving the formula above.

[jroon]

@huffyhenry

If you have divergences, then you need to resolve them first. There is a lot of advice on this forum and elsewhere how,

This is what I was trying to do with the pairs plots above 😉

@huffyhenry

Starting from the equations that you posted, it should be possible to obtain

\mathbf{y | \beta, \sigma^2, X, \tau, r, Z \sim N(X \beta, \tau K_{Z,r} + \sigma^2I)}y|β,σ2,X,τ,r,Z∼N(Xβ,τKZ,r+σ2I)\mathbf{y | \beta, \sigma^2, X, \tau, r, Z \sim N(X \beta, \tau K_{Z,r} + \sigma^2I)}

by integrating the first equation over \mathbf{h}h\mathbf{h}. I would appreciate it if someone posted an explicit derivation because I couldn’t make it work, nevertheless it is true: this result appears in Stan user’s guide §10.3 and GPML §2.2 . It is also right there at the bottom of the same page of the supplementary info from which you screenshotted the likelihood.

This is actually incredibly useful thank you. When you earlier mentioned “intergrating over”, I thought you meant a programming technique. Only with this explanation do I understand you meant actually analytically integrating the equation. So ok with your explanation and links, I see where your model code comes from above! (I am not a mathematician by training - what’s obvious for mathematicians/statisticians might be several conceptual leaps for me!)

@huffyhenry

Intuitively, BMKR posits that the (observed) outcome is the sum of the (fixed) effect of covariates \mathbf{X\beta}Xβ\mathbf{X\beta}, the (latent, random) exposure effect \mathbf{h \sim N(0, K)}h∼N(0,K)\mathbf{h \sim N(0, K)} and the observation error \mathbf{\varepsilon \sim N(0, \sigma^2I)}ε∼N(0,σ2I)\mathbf{\varepsilon \sim N(0, \sigma^2I)}. But the sum of multivariate normals is multivariate normal with mean vectors and covariance matrices added together, and adding a constant vector just shifts the mean, giving the formula above.

Ok thanks. Also helpful!