Hi everyone!

I know this forum is mostly dedicated to stan, but maybe you can help me out. I want to use loo for model ranking but my models are a jagsUI object. Also, all models are returning a warning for the pareto-k diagnostic.

In summary, I have 7 hierarchical models, each one with a single covariate. I have 115 samples and I am essentially using logistic regression to fit those models. As an output from JAGS I am storing the beta coefficients (matrix with columns as the number of coefficients and rows for each iteration) and the log likelihood (matrix with 115 columns and nrows=n_iterations).

This is the output of loo for most models

loo(model$sims.list$log.like, r_eff = NA)

Computed from 24000 by 115 log-likelihood matrix

`Estimate SE elpd_loo 1246.0 95.6 p_loo 228.3 26.6 looic 2491.9 191.3`

Monte Carlo SE of elpd_loo is NA.

`Pareto k diagnostic values: Count Pct. Min. n_eff (-Inf, 0.5] (good) 84 73.0% 1206 (0.5, 0.7] (ok) 19 16.5% 272 (0.7, 1] (bad) 6 5.2% 19 (1, Inf) (very bad) 6 5.2% 2 See help('pareto-k-diagnostic') for details. Warning message: Some Pareto k diagnostic values are too high. See help('pareto-k-diagnostic') for details.`

I checked the documentation and saw that loo_moment_match could be used to solve this issue. However, for JAGS models I would have to specify the functions for post_draws, log_lik_i, unconstrain_pars, log_prob_upars, log_lik_i_upars. I know there was a similar question previously for how to specify this for rjags objects, however, since some time has pass I was wondering if there were any new developments and maybe someone has some example code of how to specify these functions.

Sorry if this is too of topic, but I haven’t found much information for model selection and these issues when using JAGS.

Cheers!