Kfolds with upper-level unit with brms model

Modeling
1

Hi all,

Does anyone know how I can do kfolds cross-validation by leaving one upper-level unit out instead of each observation in a brms model fitted on a hierarchical dataset? loo currently does the latter but not the former.

For example, using the public cbpp dataset from package lme4, this is what I have so far.

mod3 ← brm(incidence ~ sizeR + (1 + sizeR | herd), data = cbpp)

kfold(mod3, chains = 1, group = “herd”, folds = “group”)

However, kfolds does not seem to be leaving out each upper level unit (herd), instead it’s fitting on the lower level unit (cattles).

Thank you!

2

Maybe you want following?

kfold(mod3, group = "herd", joint = "group")

My Rats case study illustrates different options. The case study is mainly using pre-defined folds to illustrate also how to use same data division if the grouping variable is not part of the model, but I also do show this specific option syntax.

2 Likes