Interpreting kfoldic in brms

I’m new to using looic, waic, and kfoldic so I have a simple question about interpreting the output of analysis:

kfoldic of model 1 = 48165, se = 233
kfoldic of model 0 = 49332, se = 225
model1 - model 0 = -1166, se = 77

Model 1 has one additional predictor over model 0. I’m wondering if given the change in ic, there is compelling reason to think that model 1 is a better model given the change in ic relative to the SE.

For sufficient number of observations, the IC values (and their differences) are approximately normally distributed witth the given SE. This helps in putting the differences into context. In your case, the difference seems to be rather substantial. You may also compute model weights via function model_weights() and weights = kfold if you want kfold to be used to compute the weights (stacking, which is the default, may be a better option though). In order to avoid recomputing kfold every single time you call kfold() I recommend taking a look at the add_ic function.

3 Likes