Interpreting output from compare() of loo

First, instead of SE, it’s better to consider something like 2SE or more cautious 4SE, where 4 comes from the fact that SE for LOO can be underestimated for small n or under bad model misspecification. Second, the models can be very different and the predictions can be very different, it’s just that the average predictive accuracies are close to each other. Third, SE describe uncertainty, so if SE is large then it’s likely that the models do have big difference in predictive accuracy, but we don’t know whether the difference is negative or positive.

7 Likes