LOO error

Hello!

I recently updated my R version to the latest including all of the packages. After the update, the LOO function does not work anymore. I have saved a workspace with estimated models and previously I was able to estimate both LOO and WAIC for all of the models but now I get the error for one model if I try to run LOO on the fitted brms object. The error is the following: “Error: Can’t mix NA and not NA values in ‘r_eff’.”.

I fitted the model again, but the problem remains.

Please also provide the following information in addition to your question:

  • Operating System: Windows 10
  • brms Version: 2.9.0

Hi, this should not be happening. Can you provide a reproducible example for me to check what’s going on?

Hi! How can I provide a reproducible example? The non-working issue (for me) seems totally random as for some models it works well but for certain estimations it gives an error. WAIC I can estimate. The output for the same model:

waic(localrd_stud)

Computed from 4000 by 5044 log-likelihood matrix

      Estimate     SE

elpd_waic -33858.4 1852.1
p_waic 939.8 121.3
waic 67716.7 3704.1
Warning message:
234 (4.6%) p_waic estimates greater than 0.4. We recommend trying loo instead.

loo(localrd_stud)
Error: Can’t mix NA and not NA values in ‘r_eff’.

Probably I am doing something wrong but re-estimating did not solve the issue. In addition, with the older version, I was able to estimate the loo for all of the brmsfit objects.

Best,
H

I had an idea. Could you try it out with the latest github version of brms to be installed via

if (!requireNamespace("devtools")) {
  install.packages("devtools")
}
devtools::install_github("paul-buerkner/brms")

Hello! Unfortunately the problem persist, I try to run the regressions once more and then come back to the issue if the problem is not solved.

Best, H

Hello!

I ran the models again. For smaller sub sample (n=594) both Loo and Waic statistics are calculated as they should. However for the second, bigger (n=6327), sample I can only compute WAIC, but not loo. The error remainds the same:

loo(localrd_stud)
Error: Can’t mix NA and not NA values in ‘r_eff’.
waic(localrd_stud)

Computed from 4000 by 6327 log-likelihood matrix

      Estimate     SE

elpd_waic -42711.8 1931.8
p_waic 992.9 100.4
waic 85423.5 3863.5

I have cleaned the dataset properly, uninstalled and installed my R and Rtools plus loaded the brms package from the provided code above.

Best, H

That’s unfortunate, but I have no way to fix this without a reproducible example.

If you like, you can send me the model producing these problems via email (paul.buerkner@gmail.com) if you don’t want to share it on discourse.

Hi Paul, I am having the same issue when I try to call the loo function through brms on my glmms with 3864 observations. The waic function still works.

Thanks @HTas for providing an example via email! I was able to fix the problem and it should now run again after installing brms from github via:

if (!requireNamespace("devtools")) {
  install.packages("devtools")
}
devtools::install_github("paul-buerkner/brms")
2 Likes