A quick note what I infer from p_loo and Pareto k values

Thanks for the code that clarified a lot. Discrete data is a bit more difficult for PIT. The current code is assuming continuous distribution. It works for count data with many values, but ZIP with many zeroes brings up the problem with count data. See Czado, Gneiting, and Held (2009). Predictive Model Assessment for Count Data. https://onlinelibrary.wiley.com/doi/full/10.1111/j.1541-0420.2009.01191.x (sorry paywalled)
Although with many zeros it’s likely that even the version for count data may have problems. In this case LOO-PPC would be easier to interpret than LOO-PIT.

@jonah I made an issue to rstantools. See the above paper p. 1255 “a nonrandomized yet uniform version of the PIT histogram”

1 Like