Posterior predictive check for binomial regression

General
#1

Hi,

There are quite few discussions how to perform posterior predictive checks (both visually and quantitatively) for binomial and logistic regression. I have a stan model for binomial classification and generated posterior predictive distribution. I am struggling to understand how to use it to assess the model.

Thanks for the advice.

Posterior Prediction from Logit Regression
#2

This paper might give you some ideas:
http://www.stat.columbia.edu/~gelman/research/published/dogs.pdf

1 Like
#3

There are examples of how to make binned and continuous calibration plots for logistic regression (made with rstanarm) at https://avehtari.github.io/modelselection/diabetes.html

2 Likes
#4

Thanks to all. It was very helpful.

I still not sure what could be discrepancy variable T in the context of veterinary trials where y is the number of dogs at a particular clinics for which treatment was successful. Can someone suggest T which makes sense?

I think dog example aligns well with veterinary trials. Each site corresponds to t in the paper while each dog corresponds to the j in the paper. Thus y_{jt} =1 if treatment was effective for a dog j at site t. Residual plots described in 4.3 are applicable. I wonder if there are some other ways to use predictive posterior for model checking in binary clinical trial domain?