Hi,
For three models as below I ran loo_compare:
p2<-get_prior( rec_n_r_bodn ~ mo(rec_comb_sevchf)+(1|idno),family=cumulative(),
data =bodi)
p2$prior[1] <- "normal(0,1)" # sets a N(0,1) on \beta
p2$prior[3] <- "normal(0,100)" # for each cutpoint
p2$prior[18] <- "cauchy(0,2.5)" # for sd
p2$prior[21] <- "dirichlet(2,2)" # sets weakly regularizing dirichlet for 1-3 Likert scale response
fit_bodn2<-brm(
formula = rec_n_r_bodn~ mo(rec_comb_sevchf)+(1|idno),family=cumulative,
data =bodi,prior=p2,chain=5,thin=1,control = list(adapt_delta =0.99,max_treedepth=15))
p3<-get_prior( rec_n_r_bodn ~ rec_comb_sevchf+(1|idno),family=cumulative(),
data =bodi)
p3$prior[1] <- "normal(0,1)" # sets a N(0,1) on \beta
p3$prior[3] <- "normal(0,100)" # for each cutpoint
p3$prior[18] <- "cauchy(0,2.5)" # for sd
fit_bodn3<-brm(
formula = rec_n_r_bodn~ rec_comb_sevchf+(1|idno),family=cumulative,
data =bodi,prior=p3,chain=5,thin=1,control = list(adapt_delta =0.99,max_treedepth=15))
p4<-get_prior( rec_n_r_bodn ~ (1|idno),family=cumulative(),
data =bodi)
p4$prior[1] <- "normal(0,1)" # sets a N(0,1) on \beta
p4$prior[16] <- "cauchy(0,2.5)" # for sd
fit_bodn4<-brm(
formula = rec_n_r_bodn~ (1|idno),family=cumulative,
data =bodi,prior=p4,chain=5,thin=1,control = list(adapt_delta =0.99,max_treedepth=15))
fit2 ← add_criterion(fit_bodn2, criterion = “loo”,reloo = TRUE)
fit3 ← add_criterion(fit_bodn3, criterion = “loo”,reloo = TRUE)
fit4 ← add_criterion(fit_bodn4, criterion = “loo”,reloo = TRUE)
loo_compare(fit2, fit3, fit4)
model_weights(fit2, fit3, fit4, weights = "loo")
fit2 fit3 fit4
9.013943e-01 9.860567e-02 6.967917e-48
loo_compare(fit2, fit3, fit4)
elpd_diff se_diff
fit2 0.0 0.0
fit3 -2.2 4.0
fit4 -108.5 15.9
Computed from 5000 by 2622 log-likelihood matrix
Estimate SE
elpd_loo -4368.7 46.4
p_loo 900.7 21.1
looic 8737.4 92.8
------
Monte Carlo SE of elpd_loo is NA.
Pareto k diagnostic values:
Count Pct. Min. n_eff
(-Inf, 0.5] (good) 2375 90.6% 263
(0.5, 0.7] (ok) 213 8.1% 89
(0.7, 1] (bad) 21 0.8% 18
(1, Inf) (very bad) 13 0.5% 5
omputed from 5000 by 2622 log-likelihood matrix
Estimate SE
elpd_loo -4366.5 46.6
p_loo 902.9 21.5
looic 8733.0 93.2
------
Monte Carlo SE of elpd_loo is NA.
Pareto k diagnostic values:
Count Pct. Min. n_eff
(-Inf, 0.5] (good) 2389 91.1% 282
(0.5, 0.7] (ok) 193 7.4% 149
(0.7, 1] (bad) 29 1.1% 18
(1, Inf) (very bad) 11 0.4% 4
See help('pareto-k-diagnostic') for details.
Can I interpret that model 2 is better as it has better weight and regarding LOO_ic model 2 and 3 are almost the same?
I read these posts as well but a bit lost in interpretation:
Thanks