Spatial model selection with loo

[cmcd]

Hey @manjago,

In my experience with spatial models, those warning from loo are completely typical. I don’t think I’ve ever fit a spatial model that did not result in those loo warnings (though I stopped checking a long time ago). Personally, I take it to mean that WAIC (and similar) might be used only as a rough indicator for spatial model comparison.

The discussion that @Carlos_Zarzar linked to is useful, I’ll just point out especially this comment from @betanalpha

WAIC unreliable?

Keep in mind that PSIS-LOO, WAIC, DIC, AIC, etc are all approximations to a fundamentally-uncalculatable number that exactly quantifies predictive accuracy. And each of those approximations require different assumptions to be reasonably accurate. A common assumption that’s necessary is that the data are roughly IID so that, for example, looking at a subset of the data gives a reasonable quantification of what a draw from the full data would do. In many models, however, such as the one that Be…

That suggests that these approximations are not well suited for spatial models: we use spatial models both because nearby observations are not IID (there is usually expected similarity), and because a draw from one side of the map is not expected to look like a draw from somewhere else.