Spatial model selection with loo

If the covariates are not correlating, you can also look at the posterior distribution of the model with all covariates.

They can be fine when comparing just a few models.

Sounds reasonable

Yes

Not necessarily. Spatial models are sometimes very flexible and require more care to assess.

2 Likes

To be precise WAIC isn’t so much an approximation of leave-one-out cross validation (of the posterior predictive density) as both WAIC and leave-one-out cross validation (of the posterior predictive density) are estimators of a common quantity: the expectation of the log posterior predictive density relative to the true data generating process.

The accuracy and precision of both estimators depends on the structure of the model. In both cases the estimation error is controlled when the observational model is comprised of independent measurements and the true data generating process is relatively normal. The error can sometimes be reasonable for more complex circumstances, but it tends to be fragile.

And then even if the estimator error is reasonable one has to consider the relevance of the object being approximated. The expectation of the log posterior predictive density has some awkward behaviors that can make it misleading at best. I go into much more detail about this object and its common estimators in Towards A Principled Bayesian Workflow.

I personally avoid these kinds of numerical quantifications and focus on visual posterior retrodictive checks – see for example Towards A Principled Bayesian Workflow. A posterior retrodictive check depends on a summary statistic to visualize the data, and the default summary statistics implemented in many software packages are often not suitable for spatial data. Consequently there will be a little bit of work needed to develop meaningful summary statistics that isolate the spatial structures of interest. Examples might include empirical correlograms or aggregated histograms of county across all spatial units (something like Towards A Principled Bayesian Workflow).

1 Like