I am trying to implement the approximate GP method you use in brms. I have it working, but I do not understand how you are making predictions.
I know with exact GPs, you need old and new data, and you need to compute the kernel using multiple covariance matrices. I perused your code, and see exactly what I expect for exact GPs.
But for approximate GPs, you say you only need the old data. Where does the ‘new’ data come in?? Obviously, it works - I can just use predict(brmsFit, newDS), and newDS only has one value in it, and it can predict using the approximate GP. Where in your code do you actually use the supplied data, and compute a predicted value? I see .predictor_gpa, but I don’t understand how we get from old-data to new-data predictions.
Oh wait - Is it literally as simple as just computing the M eigenfunctions for each row? … That may be all I need. I will check it in a bit. If so, that is way simpler than I expected. It would help if I actually understood this method beyond just implementing it.
You dont need to have the old data to compute predictions of new data for those approximate GPs. We very much hope to have a preprint of our paper soon. It is mostly shortening and that should be it.
I believe I figured it out.
You do need the gp_z values.
So, for exact GPs, you need x, x’, K(x,x), K(x,x’), K(x’,x), and K(x’,x’) (I believe). Then the rest just follows MVN expectations. From the posterior, you need alpha and rho. You can sample gp_z’s for the new data.
For the approximate GPs, you need x’, spd’s, and gp_z. From the posterior, you need alpha, rho, and gp_z. Then it’s just phi(x’)%*% (spds * gp_z), for each posterior sample of spd and gp_z. Does that sound about right?
Using that, I got very similar predictions, so it seems right.
I ‘understand’ exact GPs, but how the spectral density approx. works is beyond me. Looking forward to a preprint! The approximation works ridiculously well, and was straightforward to implement from the github notebook.
I had to look at the details of the code myself as it is some time ago I implemented it but that sounds about right. :-)