# Loo package support for Uncertainty in Bayesian leave-one-out cross-validation based model comparison

**URL:** <https://discourse.mc-stan.org/t/loo-package-support-for-uncertainty-in-bayesian-leave-one-out-cross-validation-based-model-comparison/41218>\
**Category:** Announcements\
**Tags:** loo\
**Created:** [May 13, 2026, 5:59pm UTC](https://discourse.mc-stan.org/t/loo-package-support-for-uncertainty-in-bayesian-leave-one-out-cross-validation-based-model-comparison/41218 "2026-05-13T17:59:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![avehtari](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/avehtari/32/5935_2.png) [@avehtari](https://discourse.mc-stan.org/u/avehtari)\
**Post date:** [May 13, 2026, 5:59pm UTC](https://discourse.mc-stan.org/t/loo-package-support-for-uncertainty-in-bayesian-leave-one-out-cross-validation-based-model-comparison/41218/1 "2026-05-13T17:59:49Z")

</div>

In “Uncertainty in Bayesian leave-one-out cross-validation based model comparison” [https://doi.org/10.1214/25-BA1569](https://doi.org/10.1214/25-BA1569) we showed when LOO-CV elpd\_diff and se\_diff normal approximation uncertainty quantification in model comparison is well calibrated. We (with @Florence_Bockting) have now merged a related PR to [loo package](https://mc-stan.org/loo/).

This PR adds new columns to the LOO comparison. p\_worse is the probability that a model has worse performance than the best model. diag\_diff tells whether that probability can be trusted. diag\_elpd is reminding if the single model LOO computation diagnostics are indicating issues.

It took quite long time to merge the PR as we wanted to be careful with the design choices as we had to also change from matrix output to dataframe to support text in diagnostic columns. Now the model names are also columns, making it easier for further processing and pretty printing. You can see examples of how it looks like in [LOO uncertainty case study](https://users.aalto.fi/~ave/casestudies/LOO_uncertainty/loo_uncertainty.html) by me and @Florence_Bockting

You can get this new feature by installing [loo from github](https://github.com/stan-dev/loo).

As the loo\_compare output changed from matrix to dataframe, there were few other packages that needed fixes, so if some package errors try installing their latest version. We are planning to make the next CRAN release before mid-June and then all dependent CRAN packages should work