Hi, so here’s an issue I’ve ran into in a couple analyses where simple uses of varying (random) intercepts resulted in problematic posterior distributions. In particular, the posterior for the overall intercept is negatively correlated with the posterior for all varying intercepts. The problem seem…

[image] jsocolar: I could be having a big brain fart here, but I think that what you are missing is that \gamma γ\gamma isn’t fixed but is itself random. The negative correlations in \vec{Z} →Z\vec{Z} are precisely chosen to be the right size to offset the positive correlations that are induced …

Yeah this is what I had in mind. The challenge is that it interferes a bit with our prior specification, but I think this can be dealt with using an appropriately constructed \Sigma which I am also too lazy to derive. To recover exactly the same model we need: A weaker prior on the intercept, si…

[image] martinmodrak: but I have not yet seen it discussed elsewhere. One paper discussing the issue clearly (in my opinion) is Bayesian functional {ANOVA} modeling using Gaussian process prior distributions

I’ll also note that Ogle, K., & Barber, J. J. (2020). Ensuring identifiability in hierarchical mixed effects Bayesian models. Ecological Applications. has an IMHO good discussion. I’ll read through this and Aki’s suggestion and references and hopefully come back with a summary :-)

QR Decomposition, priors on the “reverse” R^{-1} * \theta Following Stan User Manual QR reparameterization . I constructed a sum-to-zero design matrix A with intercept at the last column. > cbind(contr.sum(4),1) [,1] [,2] [,3] [,4] 1 1 0 0 1 2 0 1 0 1 3 0 0 1 1…

I am a little late to the party and @jsocolar has already said some things that I also had in mind, but since I promised to respond I will try to give my perspective anyway. First of all, I am not sure I consider the posterior correlations between overall and varying coefficients a problem per se. …

This post turned into a huge mess. I’m deleting its contents and working on an update.

It’s 3 am and I’m up reading the forums so this could be completely unhelpful, take it fwiw. The above looks a lot like a conditional MVN: Multivariate normal distribution - Wikipedia In the bivariate case it’s X_1\mid X_2=a \ \sim\ \mathcal{N}\left(\mu_1+\frac{\sigma_1}{\sigma_2}\rho( a - \mu_2)…

Interesting discussion! I just want to note here that in the Ogle et al. paper that @martinmodrak linked to, a solution is given that looks compatible with concerns expressed here about generative nature (or not) of the model. It solves the non-identifiability problem post-hoc (in the generated quan…

Ok, trying from a clean slate to write down the math here. Thanks to @andre.pfeuffer for helping to track down a bug. Given \vec{Y} \sim Normal(\alpha, \sigma), we can reparameterize using \vec{Z} = \vec{Y} - \alpha - \mu_R, where \mu_R is the sample mean of \vec{Y} - \alpha and therefore \sum{\ve…

I admit I had trouble digging through @avehtari ’s suggestion ( Kaufman & Sein, Bayesian functional ANOVA ), as they handle a very general class of models and the abstraction was a bit difficult to me, but if I understand the paper correctly, they just use a sum-to-zero constraint (and some additional …

Correlated posterior - Sum to zero constraint for varying intercepts?! - #10 by jsocolar - Modeling

Topic		Replies	Views	Activity
Sum-to-zero intercepts and terminology General	18	2763	February 29, 2024
Proper use of sum_to_zero_vector in nested multilevel models Modeling fitting-issues , specification , matrix , cmdstanr , hierarchical-model	17	852	June 2, 2025
Sum to zero constraints and multi-level models - best practise/example code Modeling	14	1394	July 16, 2025
New Stan data type: zero_sum_vector Developers specification	22	4471	April 26, 2022
Penalizing parameter to enforce a sum to 0 constraint? Modeling	13	893	May 9, 2024

Sum-to-zero intercepts and terminology

General

18

2763

February 29, 2024

Proper use of sum_to_zero_vector in nested multilevel models

Modeling

fitting-issues , specification , matrix , cmdstanr , hierarchical-model

17

852

June 2, 2025

Sum to zero constraints and multi-level models - best practise/example code

Modeling

14

1394

July 16, 2025

New Stan data type: zero_sum_vector

Developers

specification

22

4471

April 26, 2022

Penalizing parameter to enforce a sum to 0 constraint?

Modeling

13

893

May 9, 2024

Correlated posterior - Sum to zero constraint for varying intercepts?!

Related topics