I am considering a latent Gaussian process f which is observed through k different units with different intercepts b. The model is \begin{aligned} f&\sim \mathsf{GP}\left(K\right)\\ b_i&\sim\mathsf{Normal}\left(0, 1\right)\\ y_{ij}&\sim\mathsf{Normal}\left(b_i+f_j,\kappa_i^2\right), \end{aligned} …

Is there any strong reason why its preferable to enforce zero sample mean on the GP rather than zero sample mean on the intercepts?

In principle, enforcing a zero sample mean for the intercepts would also work. Are you thinking of something like b_n=-\sum_{i=1}^{n-1}b_i? My hunch is that we might observe similar behavior to 6. I’ll give that a go and report the results here (and I should share the models for reproducibility).

yep, that’s the “hard constraint” way. The “soft constraint” way is to just add something like target += normal_lpdf(sum(b) | 0, 0.001)

Sorry I cannot go through the entire list of options you have right now (may try to do it later, but… December), but I have some experience trying to fit random slopes plus a gaussian process, and not surprisingly there were identifiability issues there too. I am not aware of a real example outside …

[image] tillahoffmann: The model is As b_i have normal prior and y has normal distribution, you could include the effect of b_i in the covariance matrix K, and analytically integrate over f and b, to get stable inference for the covariance function parameters and the observation variance.

Thank you for all the suggestions! I’ve coded them up here to compare the different methods (very rough, single chain, not checking for divergences or Rhat, etc.). They are all able to infer the GP and intercepts (up to a random offset that isn’t material). In terms of runtime and reducing posterio…

[image] tillahoffmann: I have on the order of 30 units and 100 observations per unit which would leave a 3000 by 3000 covariance matrix. Probably doable but big-ish. 3000 x 3000 is not that big for GPs in general. For example, the cover of BDA3 is based on GP with 8000 x 8000 covariance matri…

Non-identifiability for Gaussian process with random intercepts

Modeling

maxbiostat December 5, 2022, 8:49am 2

@avehtari

Convergence problems in Hilbert space Gaussian process with ID effect

Topic		Replies	Views
Identifiability of Simple Gaussian Process Modeling fitting-issues , gaussian-process , reparametrization	13	2031	February 4, 2022
Likely-silly question on Gaussian Processes: can/should they be scale()'d? Modeling specification	1	811	February 10, 2022
Fitting a GP model with subject-specific GP terms in brms brms gaussian-process	4	1639	May 21, 2021
Sum-to-zero intercepts and terminology General	18	2541	February 29, 2024
Identifiability of Gaussian process Modeling	4	1070	November 24, 2021

Non-identifiability for Gaussian process with random intercepts

Related topics