I am stacking over a set of models using the loo_model_weights() function, where some of the models are fit with Stan and others are fit with JAGS. For some datasets the loo() function will give warnings that some pareto_k values are too large (> 0.7), and I would like to solve this with the loo_moment_matching() function.
I can use loo_moment_matching() for the models fit using rstan by following the vignette https://cran.r-project.org/web/packages/loo/vignettes/loo2-moment-matching.html. However, I am at a loss for how to do it with the models fit in JAGS.
One option is to recode those models in Stan, but I would prefer to keep them in JAGS because of computation time (there’s a simple Gibbs sampling algorithm that is very fast).
Any help is greatly appreciated!
I’m not sure if it will be possible to use loo_moment_match with JAGS/BUGS models since it relies on Stan’s approach to constructing parameters (mapping between a constrained and an unconstrained parameter space).
@avehtari is this something that’s been worked on before by any chance?
1 Like
I was afraid this might be the case… I’ll wait for @avehtari’s response, but in any case thanks for the quick reply!
No direct support for JAGS/BUGS. The vignette Avoiding model refits in leave-one-out cross-validation with moment matching • loo does show that you can define certain functions yourself and thus you could define those functions to evaluate the log densities. It’s not completely necessary to transform to unconstrained space, but that guarantees that constraints are not violated when making the affine transformations of the posterior draws.