Sum-to-zero intercepts and terminology

Thinking again about what I wrote above (and also given the fact that I have gained some further insight into meta-analyses), I wouldn’t say anymore that in the meta-analytic random-effects model, the sample mean is often of interest. It may sometimes be of interest, but usually, it is the population mean which is of interest, so the unconstrained model makes more sense. However, I could imagine that in case of a small number of studies included in a meta-analysis (generally speaking, a small number of groups), the sum-to-zero constrained model could be a viable alternative to the unconstrained model, analogously to what has previously been proposed in the context of MRP, see 32.5 Multilevel regression and poststratification | Stan User’s Guide. (The way I see the rationale behind this is that models are always simplifications and the question how far we need to simplify a model—or, in the other direction, how complex we can make a model—crucially depends on the data at hand. The unconstrained model can be too flexible in case of few studies and hence leads to impractically wide interval estimates for the overall tratment effect. In my opinion, an important question is always what would be the alternative approach. In meta-analyses, the commonly undertaken alternative approach is the fixed-effect—i.e., common-effect—model which is even more restrictive than the sum-to-zero constrained random-effects model.)

In general, I think @jsocolar is right that the sum-to-zero constrained model is somewhat strange from a data-generating and predictive perspective and it doesn’t really reflect the idea behind multilevel models. So in general, the unconstrained model should be the first choice (if reasonably feasible for the data at hand).

Still, I think that the pros and cons of a sum-to-zero constraint could (and perhaps should) be made clearer to users, especially with respect to the case of a small number of groups.

In any case, I think this was a very helpful discussion; thanks to everyone for participating!

4 Likes