# \#cross-validation

**URL:** https://discourse.mc-stan.org/tag/cross-validation/132.md

[Latest](https://discourse.mc-stan.org/latest.md) · [Categories](https://discourse.mc-stan.org/categories.md) · [Tags](https://discourse.mc-stan.org/tags.md)

---

## [Are stacking weights “leaking” information across folds?](https://discourse.mc-stan.org/t/are-stacking-weights-leaking-information-across-folds/40892)

<div class="topic-metadata">

**Author:** [@Federico\_Billeci](https://discourse.mc-stan.org/u/Federico_Billeci)\
**Replies:** 7\
**Last updated:** [February 10, 2026, 2:28am UTC](https://discourse.mc-stan.org/t/are-stacking-weights-leaking-information-across-folds/40892 "2026-02-10T02:28:59Z")

</div>

Hey everyone, I have a bayesian model that can be written as p(y|x, \\theta ; t ) and it is basically a logistic regression, \\theta are the bayesian parameters and t is a hyperparameter. Instead of fixing t I wanted to f…

---

## [Recent paper: Distributional bias compromises leave-one-out cross-validation](https://discourse.mc-stan.org/t/recent-paper-distributional-bias-compromises-leave-one-out-cross-validation/40743)

<div class="topic-metadata">

**Author:** [@rtnliqry](https://discourse.mc-stan.org/u/rtnliqry)\
**Replies:** 1\
**Last updated:** [December 15, 2025, 11:57am UTC](https://discourse.mc-stan.org/t/recent-paper-distributional-bias-compromises-leave-one-out-cross-validation/40743 "2025-12-15T11:57:22Z")

</div>

Hi Stan forum, An interesting paper was published recently in Science Advances that details how LOO-CV can introduce a (typically small) negative bias to model performance estimates. They propose a “rebalanced LOO-CV” t…

---

## [Model comparison for multiple imputation with brm\_multiple](https://discourse.mc-stan.org/t/model-comparison-for-multiple-imputation-with-brm-multiple/36666)

<div class="topic-metadata">

**Author:** [@jweverka](https://discourse.mc-stan.org/u/jweverka)\
**Replies:** 1\
**Last updated:** [September 20, 2024, 5:14pm UTC](https://discourse.mc-stan.org/t/model-comparison-for-multiple-imputation-with-brm-multiple/36666 "2024-09-20T17:14:16Z")

</div>

Similar issue to the one here: Using model comparison (loo or waic) after imputation, but I can’t find any discussion that is more recent than 2021, so I’d like to revive the topic. Using brms. I have a dataset with abo…

---

## [Out-of-sample cross validation for response variable with measurement error](https://discourse.mc-stan.org/t/out-of-sample-cross-validation-for-response-variable-with-measurement-error/36402)

<div class="topic-metadata">

**Author:** [@Challen\_Hyman](https://discourse.mc-stan.org/u/Challen_Hyman)\
**Replies:** 6\
**Last updated:** [September 12, 2024, 5:36pm UTC](https://discourse.mc-stan.org/t/out-of-sample-cross-validation-for-response-variable-with-measurement-error/36402 "2024-09-12T17:36:30Z")

</div>

I would ideally like to use OOS-CV to test the predictive power of a model by comparing it’s posterior predictive distributions to withheld values, but the response values have measurement error associated with them, so …

---

## [Stacked elpd\_loo estimation](https://discourse.mc-stan.org/t/stacked-elpd-loo-estimation/34916)

<div class="topic-metadata">

**Author:** [@Kjorte\_Harra](https://discourse.mc-stan.org/u/Kjorte_Harra)\
**Replies:** 4\
**Last updated:** [April 26, 2024, 8:11pm UTC](https://discourse.mc-stan.org/t/stacked-elpd-loo-estimation/34916 "2024-04-26T20:11:58Z")

</div>

Hello, I’m working on a project with Bayesian stacking of stan\_glm() linear models in rstanarm. To compare the performance of the stack to the individual member models (stored in fits\_list), I want to calculate the elpd…

---

## [Implementation of model stacking using exact leave-one-out cross-validation](https://discourse.mc-stan.org/t/implementation-of-model-stacking-using-exact-leave-one-out-cross-validation/34693)

<div class="topic-metadata">

**Author:** [@marianklose](https://discourse.mc-stan.org/u/marianklose)\
**Replies:** 5\
**Last updated:** [April 3, 2024, 4:15pm UTC](https://discourse.mc-stan.org/t/implementation-of-model-stacking-using-exact-leave-one-out-cross-validation/34693 "2024-04-03T16:15:16Z")

</div>

Hi there! I would like to implement model stacking using exact leave-one-out cross-validation for a problem at hand. However, I have a pharmaceutical/medical background and I am lacking a bit of the statistical notation …

---

## [Using \`loo\` for clustered data](https://discourse.mc-stan.org/t/using-loo-for-clustered-data/28815)

<div class="topic-metadata">

**Author:** [@avramaral](https://discourse.mc-stan.org/u/avramaral)\
**Replies:** 1\
**Last updated:** [September 12, 2022, 4:09pm UTC](https://discourse.mc-stan.org/t/using-loo-for-clustered-data/28815 "2022-09-12T16:09:27Z")

</div>

This is a more conceptual than a practical question on how to validate a fitted model. In particular how to proceed in the presence of clustered data. For instance, let y\_{ij}, such that i = 1, \\cdots, r and j = 1, \\cdo…

---

## [Is it possible to add a CV fold-dependent data-preprocessing step in \`loo::kfold()\` or the corresponding 'brms' method?](https://discourse.mc-stan.org/t/is-it-possible-to-add-a-cv-fold-dependent-data-preprocessing-step-in-loo-kfold-or-the-corresponding-brms-method/28437)

<div class="topic-metadata">

**Author:** [@henningte](https://discourse.mc-stan.org/u/henningte)\
**Replies:** 2\
**Last updated:** [August 4, 2022, 7:50am UTC](https://discourse.mc-stan.org/t/is-it-possible-to-add-a-cv-fold-dependent-data-preprocessing-step-in-loo-kfold-or-the-corresponding-brms-method/28437 "2022-08-04T07:50:07Z")

</div>

Is it possible to add a CV fold-dependent data-preprocessing step in loo::kfold() or a corresponding ‘brms’ method? I want to estimate the predictive performance of a model (computed with ‘brms’) using loo::kfold() (or …

---

## [Marginal predictions and cross-validation with latent variable model](https://discourse.mc-stan.org/t/marginal-predictions-and-cross-validation-with-latent-variable-model/25027)

<div class="topic-metadata">

**Author:** [@prototaxites](https://discourse.mc-stan.org/u/prototaxites)\
**Replies:** 17\
**Last updated:** [April 27, 2022, 9:52am UTC](https://discourse.mc-stan.org/t/marginal-predictions-and-cross-validation-with-latent-variable-model/25027 "2022-04-27T09:52:06Z")

</div>

Hi all, I am working on a generalised linear latent variable model to predict the joint abundance of different species at different sites. The model uses a factor analytic approach to allow the predicted abundances of e…

---

## [K Fold Cross Validation with Logistic Regression Model](https://discourse.mc-stan.org/t/k-fold-cross-validation-with-logistic-regression-model/27109)

<div class="topic-metadata">

**Author:** [@mdanb](https://discourse.mc-stan.org/u/mdanb)\
**Replies:** 4\
**Last updated:** [April 12, 2022, 7:57am UTC](https://discourse.mc-stan.org/t/k-fold-cross-validation-with-logistic-regression-model/27109 "2022-04-12T07:57:45Z")

</div>

I’m following along with this vignette to perform K-fold cross validation for a logistic regression model. I’ve included my model code at the end of this post. At one point, the author does this: fit \<- sampling(stanm…

---

## [Multiple attemps to run k-fold cross-validation fail with brms](https://discourse.mc-stan.org/t/multiple-attemps-to-run-k-fold-cross-validation-fail-with-brms/25684)

<div class="topic-metadata">

**Author:** [@Maxime](https://discourse.mc-stan.org/u/Maxime)\
**Replies:** 0\
**Last updated:** [December 14, 2021, 10:53am UTC](https://discourse.mc-stan.org/t/multiple-attemps-to-run-k-fold-cross-validation-fail-with-brms/25684 "2021-12-14T10:53:59Z")

</div>

Good day, I have been trying to run k-fold cross-validation on a brms model without success. Unfortunately, it is difficult to provide a reproducible example because my model is run on a large dataset that is private. I…

---

## [If elpd\_diff/se\_diff \> |2|, is this noteworthy?](https://discourse.mc-stan.org/t/if-elpd-diff-se-diff-2-is-this-noteworthy/20549)

<div class="topic-metadata">

**Author:** [@llewmills](https://discourse.mc-stan.org/u/llewmills)\
**Replies:** 20\
**Last updated:** [February 8, 2021, 12:35am UTC](https://discourse.mc-stan.org/t/if-elpd-diff-se-diff-2-is-this-noteworthy/20549 "2021-02-08T00:35:36Z")

</div>

I am conducting a forward iterative model comparison analysis of longitudinal data, similar to that discussed by Singer and Willett in Applied Longitudinal Data Analysis but using loo-cv methods in stan/brms. Excuse the …

---

## [Arviz.reloo: am I doing it right?](https://discourse.mc-stan.org/t/arviz-reloo-am-i-doing-it-right/20988)

<div class="topic-metadata">

**Author:** [@Teddy\_Groves1](https://discourse.mc-stan.org/u/Teddy_Groves1)\
**Replies:** 2\
**Last updated:** [February 25, 2021, 8:19pm UTC](https://discourse.mc-stan.org/t/arviz-reloo-am-i-doing-it-right/20988 "2021-02-25T20:19:11Z")

</div>

I’m continuing a discussion from arviz github with @OriolAbril here for better exposure. I would like to compare the out-of-sample predictive performance of some statistical models that I’ve been fitting and analysing w…

---

## [How to speed up GLMM fitting when drawing predictions for a hold out data set?](https://discourse.mc-stan.org/t/how-to-speed-up-glmm-fitting-when-drawing-predictions-for-a-hold-out-data-set/20606)

<div class="topic-metadata">

**Author:** [@OBidz](https://discourse.mc-stan.org/u/OBidz)\
**Replies:** 2\
**Last updated:** [February 8, 2021, 6:09pm UTC](https://discourse.mc-stan.org/t/how-to-speed-up-glmm-fitting-when-drawing-predictions-for-a-hold-out-data-set/20606 "2021-02-08T18:09:24Z")

</div>

Below is a GLMM, with intercepts and beta parameters that differ by group (no pooling), and a sigma parameter in the call to normal() that also differs by group. Fighting with the syntax for the X \* B vector multiplicati…

---

## [New reduce\_sum makes cross-validation simple; should we standardise?](https://discourse.mc-stan.org/t/new-reduce-sum-makes-cross-validation-simple-should-we-standardise/14332)

<div class="topic-metadata">

**Author:** [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Replies:** 0\
**Last updated:** [April 15, 2020, 10:00am UTC](https://discourse.mc-stan.org/t/new-reduce-sum-makes-cross-validation-simple-should-we-standardise/14332 "2020-04-15T10:00:40Z")

</div>

Hi! The 2.23 release is bringing us reduce\_sum which enables efficient within-chain parallelisation in Stan in a scalable way. I am relatively sure that this will change the way we write our models whenever these have t…