Interpreting elpd_diff - loo package

[avehtari]

@Benambridge

What I’m trying to do is compare model a (a baseline model) and model b:
a=(Un~ (1 + Entrenchment|PID) + (1|Verb), data=BOTH)
b=(Un~ (1 + Entrenchment|PID) + (1|Verb) + Entrenchment, data=BOTH)

I repeat that this comparison is conceptually very easy with k-fold-cv, and now your problem is just that you would need to write a few lines of code to do the division of the data according to PID-group structure.

In the future, we’ll add support to rstanarm so that you can do the following very easily

fit1<-stan_glmer(Un~ (1 + Entrenchment|PID) + (1|Verb), data=BOTH)
fit2<-stan_glmer(Un~ (1 + Entrenchment|PID) + (1|Verb) + Entrenchment, data=BOTH)

(kfold1 ← kfold(fit1, groupby = PID)) #not yet implemented
(kfold2 ← kfold(fit2, groupby = PID)) #not yet implemented
compare(kfold1, kfold2)

I’m sorry, but we don’t have resources to add these quicker.