Error adding Loo criterion to brms ordinal model with link='cloglog'

Interfaces brms
1

The model looks fine but when I try to add loo or waic criterion I get the error
validate_ll(x) : All input values must be finite

I’m not convinced that the cloglog link is appropriate. I get fairly good results using the ‘logit’ link. The data are very skewed so I was hoping to check. I’m fairly confident that the sratio model is a good model of the data generating process.

df <- read.csv("test_error_validate_ll_ordinal.csv")
df[,'wp10'] <- factor(df[,'wp10'],levels=c("stub",'start','c','b','ga','fa'),ordered=T)
fam.cloglog <- sratio(link='cloglog', threshold='flexible')
formula <- brmsformula(wp10 | weights(weight) ~  Stub + Start + B + FA + GA, decomp='QR',center=TRUE)
mod <- brm(formula=formula.main.noC, data=df, family=fam.cloglog)
mod <- add_criterion(model.main.noC.cloglog,'loo')

I have uploaded my dataset (n=2000, 258kb) here: https://teblunthuis.cc/outgoing/test_error_validate_ll_ordinal.csv

  • Operating System: centos 7.6.1810
  • brms Version: 2.15.0

Thank you so much for the help!

2

It’s been awhile since I asked this question. Can anybody help?

3

I assume this is caused by numerical instability of common for the cloglog link. Recently, we have made some change to the brms code-base relating to this part. We did not explicitly improve numeric stability but we perhaps we have “accidentally” made things better. Could you try again with the latest brms version from github?

4

Thanks! I just checked and got the same error.

5

Yeah, it’s definitely a numerical issue. We would have to compute stuff on the log-scale as we do already in the corresponding Stan code. I don’t have time to fix this right now unfortunately, but would you mind opening an issue on GitHub - paul-buerkner/brms: brms R package for Bayesian generalized multivariate non-linear multilevel models using Stan pointing to this thread so that this issue is not forgotten?