Comparison of two model BYM and BYM2

Without a reproducible example, we can only speculate about exactly what is going wrong, but the first thing that I notice is that in

vector[N] log_lik;
  for (n in 1:N) {
    log_lik[n] = poisson_log_lpmf(y[n] | log_E + beta0 + x[n] * betas + convolved_re * sigma);
  }

you seem to be missing an index [n] on convolved_re. Perhaps this propagates forward to the warning that you see from the loo package.

However, @maxbiostat is rightly raising a different point, which is that you probably shouldn’t expect PSIS-LOO to work for your model. For clarity, we can distinguish three cases:

  • PSIS-LOO where the withhold consists of one row and there are many rows per spatial region in your model. This is likely to work, but the code you’ve provided indicates that you have just one observation per region, so this is not your use case.
  • PSIS-LOO where the withhold consists of an entire region, either because there is one data row per region or because out-of-sample prediction to new regions is of interest. It’s likely PSIS-LOO won’t perform well when the withhold removes all of the data informing a random effect parameter, and this is the situation that @maxbiostat is rightly thinking about.
  • Actual brute-force cross-validation, which will in general be valid even when the PSIS-LOO approximation fails. My guess is that this is what you will ultimately want to use here.

There’s a nice discussion of LOO in the context of spatial models here Spatial model selection with loo