Pareto values versus ELPD differences

[jsocolar]

@avehtari could provide a better answer than I, but in general high Pareto k values in either model mean that you shouldn’t trust LOO’s model comparison. But it is possible for either or both models to be properly specified despite high Pareto k values. You might find this post helpful:

A quick note what I infer from p_loo and Pareto k values

This a quick note on different cases why Pareto k values can be large. I’ll try to extend this to a longer explanation and a case study. Any feedback on this draft is appreciated If all Pareto k small, model is likely to be ok (although there can be better models) If high Pareto k values If p_loo << the number of parameters p, then the model is likely to be misspecified. PPC is likely to detect the problem, too. Try using overdispersed model, or add more structural information (nonlinearity, …