How do I interpret this loo-pitt plot?
My other model diagonistic predictions were reasonable:
So I don’t know why this is acting weird.
I have answered this question in your earlier thread Model Selection in BRMS - #13 by avehtari (it would be nice for others to mention that this has already been discussed so far, so that they could check the discussion so far). The answer is still the same: “LOO-PIT can fail for discrete data, although it seems to failing in a way that shouldn’t happen.” If that sentence was unclear, I now clarify that the interpretation is that the computation for the plot is failing, and we can’t make other interpretations.
I don’t think anyone knows why it is that weird, showing something that shouldn’t be possible. We do know that the current code does not support discrete data (which you have), but even then I would not expect that weird plot.
Can you provide the LOO-PIT values? Even better if you can provide the full example.
1 Like