Identifiability of Simple Gaussian Process

Modeling
, ,
10

Oh! I see now where and why I have lost you! I didn’t make the connection to that previous post (of yours) Hilbert space approximate GP prior: why use de-meaned basis functions (PHI)?

I see why this has been confusing 😅

Anyways, when I talked about the centeredness of the parameterization I wasn’t talking about demeaning the basis functions. “Centered” / “Non-centered” parameterizations are the common but very unhelpful names for two specific ways of parametrizing hierarchical models, see eg 24.7 Reparameterization | Stan User’s Guide But of course in your last post everybody meant demeaned / non-demeaned basis functions when saying centered / non-centered.

I wanted to write a something up anyways, but this may take a few days in which I figure out the details myself. In the meantime the above link may be interesting, if not necessarily helpful for your current problem.

Edit:

Yes.

Yes, it’s only a change of variables.

Yes, they should still be normal after reparametrization (but I don’t know wheter this is always necessary for GP discretizations).

In an ideal world, the so called centered parametrization (for a simpler model) would be

parameters {
   real<lower=0> sigma;
   real x;
}
model {
   sigma ~ lognormal(0,1);
   x ~ normal(0, sigma);
}

and the non-centered one would be

parameters {
   real<lower=0> sigma;
   real<offset=0, multiplier=sigma> x;
}
model {
   sigma ~ lognormal(0,1);
   x ~ normal(0, sigma);
}

which internally would look something like this

parameters {
   real<lower=0> sigma;
   real xi;
}
transformed parameters {
  real x = sigma * xi;
}
model {
   sigma ~ lognormal(0,1);
   xi ~ normal(0, 1);
}

But I think there is some bug in the implementation of the offset/multiplier implementation where it sometimes gets stuck in weird places for numerical reasons.

2 Likes