Identifiability of Gaussian process

Modeling
1

Hi community,

I have difficulty in understanding the last section of 10.3 Fitting a Gaussian process | Stan User’s Guide, i.e. Multiple-output Gaussian processes.

Screenshot 2021-11-23 at 16.06.15
Screenshot 2021-11-23 at 16.06.15850×505 33.7 KB

In the screenshot, we need to set \alpha=1 to make parameters identifiable.

In the Stan code, \alpha is taken out accordingly by
f = L_K * eta * diag_pre_multiply(alpha, L_Omega)’

model {
  matrix[N, D] f;
  {
    matrix[N, N] K = cov_exp_quad(x, 1.0, rho);
    matrix[N, N] L_K;

    // diagonal elements
    for (n in 1:N)
      K[n, n] = K[n, n] + delta;

    L_K = cholesky_decompose(K);
    f = L_K * eta
        * diag_pre_multiply(alpha, L_Omega)';
  }

Sorry that I don’t quite understand why taking \alpha outside of K will make the GP identifiable. And I cannot see the differences made to identifiability. Could anyone explain?

Thank you so much in advance!

EDIT: changed the link to point to the latest guide version (which has several typos fixed)

2

@avehtari @betanalpha

1 Like
3

It looks like an unfortunate switch of variable names, and alpha in the code is part of C which is not explicitly constructed in the code (and thus alpha in the code is not the same as alpha in the equations). It would be good to edit that section and code to make it easier to compare equations and code.

2 Likes
4

Thank you so much for your reply! But I’m still not sure why does moving \alpha outside of K and incorporating \alpha into C help with the model identifiability?

5

We can consider that both K and C have their on alpha:
K(\alpha_K,\rho_K)C(\alpha_C,\eta_C)=\alpha_K \alpha_C K(1,\rho_K)C(1,\eta_C)
You can identify the product \alpha_K \alpha_C, but not \alpha_K and \alpha_C separately. So we can fix one of them. Now as \alpha_C is a vector, it is natural to fix the scalar \alpha_K=1.

2 Likes