Spatial model selection with loo

[Carlos_Zarzar]

Hello @manjago.
Regarding your first question.

@manjago

Firts, I get the warning about p_waic values greater tha 0.4 and no matter how many iterations I use, the standard errors maintains constant so I can determine which one is the best, even if the warning is ignored.

Both p_waic and “its equivalent” p_loo are estimates of the effective number of parameters in your proposed model. It is recommended that this estimate be close to the true number of parameters to be estimated in the model. And clearly, that involves sample sizes and every statistical principle.

When you get the warning about p_waic values greater than 0.4, the diagnosis is not reliable (and 0.4 threshold is empirically chosen). Aki Vethari (the responsible for that specific warning message) recommends using PSIS-LOO (Here you can find a quick overview of p_loo and Pareto K-values).

In this topic here in the Forum and in this discussion site you will find more interesting information for your problem.

@manjago

If I use loo then I get the warning about the pareto values greater than 0.7 so finally I use kfold and get an error about the limitation of new locations in CAR models.

If I’m not mistaken. Well summarized. Pareto K values in addition to indicating whether the model you are proposing is well or poorly specified for the phenomenon under study. It can give clues to influential points and leverage points. So this may be happening in your case (although I haven’t run your model with your data to confirm this).

• Some references if you want to delve into the theory a little more are below.

1. Vehtari, A., Simpson, D., Gelman, A., Yao, Y., & Gabry, J. (2015). Pareto smoothed importance sampling. arXiv preprint arXiv:1507.02646 .;

2. Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and computing , 27 (5), 1413-1432.

Hope this helps.