Pareto values versus ELPD differences

This post is also now part of the loo documentation LOO package glossary — loo-glossary • loo

Without not yet knowing the number of parameters, just knowing that p_loo is 35-43% from the number of observations we can infer that it’s likely that the model is flexible. If p_loo is higher than the number of parameters then the model is certainly misspecified, but if p_loo is lower than the number of parameters I would expect that both models are flexible, which could mean, e.g. a hierarchical model with group specific parameters but not many observations for some groups. If p_loo is less than the number of the parameters and other model checking diagnostics don’t indicate bad model misspecification then the difference between the models is that big that you can say that the model 2 has better predictive performance.

Check also my yesterday post Model selection of nonlinear flexible hierarchical model with loo - #2 by avehtari

1 Like