I have response categories 1,2,3,4 and a list of predictors. I want to use cross-validation to test my model fit. I’m doing cross validation, rather than loo because I need to leave out all instances belonging to the same participant. This is the first time of me using cross-validation with this type of model, so I would really appreciate if I could get some advice on whether I’m doing the right thing.

Here is the model I’ve used. I did not include a hierarchical term - each participant has the same 4 data points (1 of each of the 4 categories) and I have removed for each predictors the average within each person (because across people the averages were very different). I did this because I’m not interested in the regression weights (e.g. whether they are significant), but only in whether the overall model has predictive power in the test set.

```
my.formula=as.formula('Categs ~ 1+ a+ b+ c+ d) # to give the gist
fit = brm(formula=my.formula,
data=df,
prior=my.priors,
iter=4000,control=list(adapt_delta=0.9),
family=cumulative(link='probit'))
```

And then I tried to implement k-fold cross-validation like this

```
kf = kfold(fit,folds=df$FoldID)
```

Where df$FoldID is for each entry which fold they belong to (I’ve generated that manually to make sure I’ve respected the grouping). [for now I just had two participants, but of course will have more…]

I get a result like this:

```
Based on 2-fold cross-validation
Estimate SE
elpd_kfold -6.6 0.2
p_kfold 0.0 0.1
kfoldic 13.2 0.5
```

I am now not quite sure how to know what this error means. How is the error computed for 4 monotonic groups? When I tried running predictive_error, it said “Error: Predictive errors are not defined for ordinal or categorical models.” But then I’m not sure what the k-fold cross-validation is computing?

I’m thinking I’ll compute the same for an empty model (intercept only) and then compare the models?