New tutorial: Cross-validation for hierarchical/multilevel models

People have been asking for something like for long time, and finally I have the first version of a tutorial for how to do cross-validation for hierarchical/multilevel models using rstanarm, loo and kfold-helper functions. The html rendered version is https://avehtari.github.io/modelselection/rats_kcv.html.

To run the tutorial code you need the latest rstanarm and loo version from github. It an take a few hours to run the whole tutorial (there are many K-fold-CV runs), but you can pick the relevant parts for your own experiments. Although the tutorial is using R and rstanarm, the discussion in the tutorial should be helpful for non-R-users, too. There will be a brms version later.

25 Likes