Adding Gaussian Process Covariance Functions

[betanalpha]

@Bob_Carpenter

Also keep in mind that people who want to build more complicated models tend to be able to deal with more complexity—it’s self selecting. So slight burdens for combination wouldn’t be too bad, and arguably better than putting a slight burden on single-kernel usage.

Excellent point.

@Bob_Carpenter

If we know what this single-kernel use should look like, then I don’t see a problem with building that first.

We do – see for example all of the examples in my GP case studies,
https://betanalpha.github.io/assets/case_studies/gp_part1/part1.html
https://betanalpha.github.io/assets/case_studies/gp_part2/part2.html
https://betanalpha.github.io/assets/case_studies/gp_part3/part3.html

These use cases motivated the original design spec on the wiki.

@Bob_Carpenter

As far as I know, there isn’t even a proposal on the table yet for combining kernels in an interesting way. If you want to think of kernels as functions, it may work out better when we have functional types in a year or so (now I’m being optimistic!).

There isn’t a full proposal but there is the immediate decision of “do we pack together scaling hyperparameters and jitter/nuggets which makes single kernels slightly easier to implement” or “do we define kernels independent of these hyperparameters which makes it easier to combine kernels together and hence facilitating whatever kernel algebra library might be introduced in the future”.

@anon75146577

As Aki said: nugget and jitter are two very different things. If for no other reason than one is random and one is deterministic.

Nugget only exists in spatial stats because the old guard don’t like hierarchical models and focussed strongly in Gaussian observation noise. It has no role in modern life (ie it should be somewhere else in the model rather than the covariance function)

When I was writing the GP case studies I was definitely yelled at to refer to “nugget” as the small diagonal contribution to stabilize the calculations even when there isn’t a conjugate Gaussian observation. Regardless I’m happy to go with jitter if it won’t upset the spatial stats people.

Regardless, if the diagonal term is templated out then it can used for jitter (by dumping a constant in) or unknown measurement variability (by dumping a parameter in).