I am using a binomial GLMM and am confused on how to test for homoscedasticity of residuals.
When I use normal difference residuals it shows heteroscedasticity, but when I use pearsons residuals it shows more of a homoscedastic trend.
I don’t know how to use deviance residuals, but red that ppc_check has similar meaning.
So my question is what to do when testing the assumptions for Binomial GLMM
@paul.buerkner
You might find this thread useful:
To be nitpicky about vocabulary for a moment, note that you do not expect homoscedasticity of residuals in a binomial model. For a binomial response, the analog to checking residuals in a Guassian model for normality and homoscedasticity is checking the binomial response for over- (or under-) dispersion.
Wow this response was very helpful thanks @jsocolar
Ok so when checking for (over or under- ) dispersion the posterior predictive check can give you insight into this based on how the predictions are distributed around your observed values.
So my follow up question is if your model is showing more under-dispersion than over (meaning the predicted values tend to be lower than the observed values), then what are some techniques to correct for this.
Under-dispersion is not the same thing as predicted values tending to be lower than observed values. Under-dispersion is if predicted values tend to be more variable than observed values (the observed values are under-dispersed compared to binomial sampling).
If you are seeing a clear systematic tendency for underestimation of the observed values, then it is likely that you have prior-data conflict (otherwise, the model would be free to adjust the intercept parameter until it was getting things right on average). If you are seeing a clear tendency to underestimate the values for a subset of points (as in, the posterior predictive distribution completely fails to cover their values, or it mostly fails to cover the values for a lot of points), then it is likely that you have overdispersion instead. That is, binomial sampling variation is too low to reflect the variation contained in your data. In this case, you would expect to see systematic underestimation of some points and systematic overestimation of other points, beyond what should be expected from binomial sampling variation. Remedies include adding more covariates (but only if they actually have explanatory power) and adding more flexible random effects structures, up to and including an observation-level random effect (this can work well when at least some observations involve more than a couple of trials; it leads to non-identifiability if the observations are binary (i.e. one trial each; Bernoulli sampling variation).
Binomial Logistic GLMM
Binary Logistic GLMM
These are my posterior predictive checks. So would these indicate over-dispersion? @jsocolar
These plots contain no indication of misspecification, but they are not the most sensitive tests for over- or under-dispersion.
What test would you recommend for under or over-dispersion?
If you’re interested in logistic regression from a residual perspective, Roback and Legler cover that in chapter 6. From a pp-check perspective, those plots look spectacular.
Thanks for the reference. Would you recommend any other sources for testing model assumptions and understanding different pp_checks.
Also @Solomon I would really appreciate it if I could get your input on this question.
You’re asking a big question, which requires a similarly big answer. Consider this book, Statistical Rethinking | Richard McElreath, and its related lecture series, https://www.youtube.com/@rmcelreath/playlists. I can’t overstate how big of a change it is to move away from a conventional ‘test the residuals’ paradigm to a ‘check the posterior predictive distribution to ensure the model is validly answering my research question’ type paradigm.
As to the link you provided, I’m not experienced with mult-membership, so I’ll have to let others chip in.
Wow thanks I really appreciate the help.
Hello! By any chance, do you have any papers or resources that explain why we don’t check for homoscedasticity in binomial models and what the proper diagnostics are?
We don’t check homoscedasticity in binomial models because binomial models don’t make that assumption. Any good GLM textbook will cover this. Here are three:
@isabelaruelas, it looks like this is your first post. Welcome to the Stan forums!
thank you so much for the super quick answer! :D
