any particular method for which it fails?
I canβt say for sure, but I think the trend in post-processing is that functions calling to some form of posterior predictives vs posterior samples result in the crash. Most readily I find that pp_check()
and add_criterion()
are the biggest culprits, but those are also the post-processing things I call the most frequently so that might just be selection bias. I know that doesnβt explain the crashes that sometimes happen after sampling or right during compiling, but those happen very rarely for me and almost always itβs because I took the shortcut of not deleting an old fit before running the model again in the same session.
In my mind, the posterior predictions being a cause for Rstudio crashes makes sense if it is a memory commitment issue and Rstudio having a low threshold for crashing. Itβs been a while now, but I recall seeing discussions on CrossValidated discussing the seemingly arbitrary memory allotment for Windows + Rstudio. My guess (which I really make with a grain of salt because Iβm talking out the wrong end of me on these kinds of topics) is that generating those predictions and manipulating sometimes very large arrays probably produces some kind of spike in computing needs that trips something in Rstudio. Possibly, setting mc.cores to less than the total available helps avert Rstudio from believing that its going to over commit the computerβs resources (?)
Yeah. It might indeed be a problem with the default of using more than 1 core on windows in some post-processing functions (Change default `cores` for `posterior_predict` on Windows Β· Issue #1129 Β· paul-buerkner/brms Β· GitHub). I will change that later on today and let you know once its done.
Just fixed it in the github version of brms. If the crashes are because of accidental use of multiple cores in post-processing functions on windows, it should no longer occur.
Iβm having a similar issue using Ubuntu 20.04. I can fit models with brms, but then using post-processing functions creates crashes. After updating with the github version I could get the first plot (but not the second), and now RStudio crashes always when executing the first plot function.
library(brms)
#> Loading required package: Rcpp
#> Loading 'brms' package (version 2.15.5). Useful instructions
#> can be found by typing help('brms'). A more detailed introduction
#> to the package is available through vignette('brms_overview').
#>
#> Attaching package: 'brms'
#> The following object is masked from 'package:stats':
#>
#> ar
options(mc.cores = parallel::detectCores())
fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient),
data = epilepsy, family = poisson())
#> Compiling Stan program...
#> Start sampling
summary(fit1)
#> Family: poisson
#> Links: mu = log
#> Formula: count ~ zAge + zBase * Trt + (1 | patient)
#> Data: epilepsy (Number of observations: 236)
#> Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#> total post-warmup samples = 4000
#>
#> Group-Level Effects:
#> ~patient (Number of levels: 59)
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sd(Intercept) 0.59 0.07 0.46 0.74 1.00 1010 1634
#>
#> Population-Level Effects:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept 1.77 0.11 1.54 1.99 1.01 755 1643
#> zAge 0.10 0.09 -0.08 0.26 1.00 821 1188
#> zBase 0.70 0.12 0.46 0.93 1.00 767 1434
#> Trt1 -0.27 0.17 -0.59 0.06 1.00 812 1692
#> zBase:Trt1 0.06 0.16 -0.28 0.38 1.00 917 1492
#>
#> Samples were drawn using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).
plot(conditional_effects(fit1, effects = "zBase:Trt"))
plot(fit1, pars = c("Trt", "zBase"))
sessioninfo::session_info()
#> β Session info βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
#> setting value
#> version R version 4.1.0 (2021-05-18)
#> os Ubuntu 20.04.2 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz Europe/Berlin
#> date 2021-05-26
#>
#> β Packages βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
#> package * version date lib source
#> abind 1.4-5 2016-07-21 [1] CRAN (R 4.1.0)
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.1.0)
#> backports 1.2.1 2020-12-09 [1] CRAN (R 4.1.0)
#> base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.1.0)
#> bayesplot 1.8.0 2021-01-10 [1] CRAN (R 4.1.0)
#> boot 1.3-28 2021-05-03 [4] CRAN (R 4.0.5)
#> bridgesampling 1.1-2 2021-04-16 [1] CRAN (R 4.1.0)
#> brms * 2.15.5 2021-05-26 [1] Github (paul-buerkner/brms@fdd776c)
#> Brobdingnag 1.2-6 2018-08-13 [1] CRAN (R 4.1.0)
#> callr 3.7.0 2021-04-20 [1] CRAN (R 4.1.0)
#> cli 2.5.0 2021-04-26 [1] CRAN (R 4.1.0)
#> coda 0.19-4 2020-09-30 [1] CRAN (R 4.1.0)
#> codetools 0.2-18 2020-11-04 [4] CRAN (R 4.0.3)
#> colorspace 2.0-1 2021-05-04 [1] CRAN (R 4.1.0)
#> colourpicker 1.1.0 2020-09-14 [1] CRAN (R 4.1.0)
#> crayon 1.4.1 2021-02-08 [1] CRAN (R 4.1.0)
#> crosstalk 1.1.1 2021-01-12 [1] CRAN (R 4.1.0)
#> curl 4.3.1 2021-04-30 [1] CRAN (R 4.1.0)
#> DBI 1.1.1 2021-01-15 [1] CRAN (R 4.1.0)
#> digest 0.6.27 2020-10-24 [1] CRAN (R 4.1.0)
#> dplyr 1.0.6 2021-05-05 [1] CRAN (R 4.1.0)
#> DT 0.18 2021-04-14 [1] CRAN (R 4.1.0)
#> dygraphs 1.1.1.6 2018-07-11 [1] CRAN (R 4.1.0)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.1.0)
#> emmeans 1.6.0 2021-04-24 [1] CRAN (R 4.1.0)
#> estimability 1.3 2018-02-11 [1] CRAN (R 4.1.0)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.1.0)
#> fansi 0.5.0 2021-05-25 [1] CRAN (R 4.1.0)
#> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.1.0)
#> fs 1.5.0 2020-07-31 [1] CRAN (R 4.1.0)
#> gamm4 0.2-6 2020-04-03 [1] CRAN (R 4.1.0)
#> generics 0.1.0 2020-10-31 [1] CRAN (R 4.1.0)
#> ggplot2 3.3.3 2020-12-30 [1] CRAN (R 4.1.0)
#> ggridges 0.5.3 2021-01-08 [1] CRAN (R 4.1.0)
#> glue 1.4.2 2020-08-27 [1] CRAN (R 4.1.0)
#> gridExtra 2.3 2017-09-09 [1] CRAN (R 4.1.0)
#> gtable 0.3.0 2019-03-25 [1] CRAN (R 4.1.0)
#> gtools 3.8.2 2020-03-31 [1] CRAN (R 4.1.0)
#> highr 0.9 2021-04-16 [1] CRAN (R 4.1.0)
#> htmltools 0.5.1.1 2021-01-22 [1] CRAN (R 4.1.0)
#> htmlwidgets 1.5.3 2020-12-10 [1] CRAN (R 4.1.0)
#> httpuv 1.6.1 2021-05-07 [1] CRAN (R 4.1.0)
#> igraph 1.2.6 2020-10-06 [1] CRAN (R 4.1.0)
#> inline 0.3.18 2021-05-18 [1] CRAN (R 4.1.0)
#> jsonlite 1.7.2 2020-12-09 [1] CRAN (R 4.1.0)
#> knitr 1.33 2021-04-24 [1] CRAN (R 4.1.0)
#> later 1.2.0 2021-04-23 [1] CRAN (R 4.1.0)
#> lattice 0.20-44 2021-05-02 [4] CRAN (R 4.1.0)
#> lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.1.0)
#> lme4 1.1-27 2021-05-15 [1] CRAN (R 4.1.0)
#> loo 2.4.1 2020-12-09 [1] CRAN (R 4.1.0)
#> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.1.0)
#> markdown 1.1 2019-08-07 [1] CRAN (R 4.1.0)
#> MASS 7.3-54 2021-05-03 [4] CRAN (R 4.0.5)
#> Matrix 1.3-3 2021-05-04 [4] CRAN (R 4.0.5)
#> matrixStats 0.58.0 2021-01-29 [1] CRAN (R 4.1.0)
#> mgcv 1.8-35 2021-04-18 [4] CRAN (R 4.0.5)
#> mime 0.10 2021-02-13 [1] CRAN (R 4.1.0)
#> miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 4.1.0)
#> minqa 1.2.4 2014-10-09 [1] CRAN (R 4.1.0)
#> multcomp 1.4-17 2021-04-29 [1] CRAN (R 4.1.0)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.1.0)
#> mvtnorm 1.1-1 2020-06-09 [1] CRAN (R 4.1.0)
#> nlme 3.1-152 2021-02-04 [4] CRAN (R 4.0.3)
#> nloptr 1.2.2.2 2020-07-02 [1] CRAN (R 4.1.0)
#> pillar 1.6.1 2021-05-16 [1] CRAN (R 4.1.0)
#> pkgbuild 1.2.0 2020-12-15 [1] CRAN (R 4.1.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.0)
#> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.1.0)
#> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.1.0)
#> processx 3.5.2 2021-04-30 [1] CRAN (R 4.1.0)
#> projpred 2.0.2 2020-10-28 [1] CRAN (R 4.1.0)
#> promises 1.2.0.1 2021-02-11 [1] CRAN (R 4.1.0)
#> ps 1.6.0 2021-02-28 [1] CRAN (R 4.1.0)
#> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.1.0)
#> R6 2.5.0 2020-10-28 [1] CRAN (R 4.1.0)
#> Rcpp * 1.0.6 2021-01-15 [1] CRAN (R 4.1.0)
#> RcppParallel 5.1.4 2021-05-04 [1] CRAN (R 4.1.0)
#> reprex 2.0.0 2021-04-02 [1] CRAN (R 4.1.0)
#> reshape2 1.4.4 2020-04-09 [1] CRAN (R 4.1.0)
#> rlang 0.4.11 2021-04-30 [1] CRAN (R 4.1.0)
#> rmarkdown 2.8 2021-05-07 [1] CRAN (R 4.1.0)
#> rsconnect 0.8.18 2021-05-24 [1] CRAN (R 4.1.0)
#> rstan 2.26.1 2021-05-26 [1] local
#> rstantools 2.1.1 2020-07-06 [1] CRAN (R 4.1.0)
#> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.1.0)
#> sandwich 3.0-1 2021-05-18 [1] CRAN (R 4.1.0)
#> scales 1.1.1 2020-05-11 [1] CRAN (R 4.1.0)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.1.0)
#> shiny 1.6.0 2021-01-25 [1] CRAN (R 4.1.0)
#> shinyjs 2.0.0 2020-09-09 [1] CRAN (R 4.1.0)
#> shinystan 2.5.0 2018-05-01 [1] CRAN (R 4.1.0)
#> shinythemes 1.2.0 2021-01-25 [1] CRAN (R 4.1.0)
#> StanHeaders 2.26.1 2021-05-26 [1] local
#> stringi 1.6.2 2021-05-17 [1] CRAN (R 4.1.0)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.1.0)
#> survival 3.2-11 2021-04-26 [4] CRAN (R 4.0.5)
#> TH.data 1.0-10 2019-01-21 [1] CRAN (R 4.1.0)
#> threejs 0.3.3 2020-01-21 [1] CRAN (R 4.1.0)
#> tibble 3.1.2 2021-05-16 [1] CRAN (R 4.1.0)
#> tidyselect 1.1.1 2021-04-30 [1] CRAN (R 4.1.0)
#> utf8 1.2.1 2021-03-12 [1] CRAN (R 4.1.0)
#> V8 3.4.2 2021-05-01 [1] CRAN (R 4.1.0)
#> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.1.0)
#> withr 2.4.2 2021-04-18 [1] CRAN (R 4.1.0)
#> xfun 0.23 2021-05-15 [1] CRAN (R 4.1.0)
#> xtable 1.8-4 2019-04-21 [1] CRAN (R 4.1.0)
#> xts 0.12.1 2020-09-09 [1] CRAN (R 4.1.0)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.1.0)
#> zoo 1.8-9 2021-03-09 [1] CRAN (R 4.1.0)
#>
#> [1] /home/cmoya/R/x86_64-pc-linux-gnu-library/4.1
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library
It seems to be a problem on the RStudio side. The code runs if directly executed in the terminal. It seems to be fixed by using the latest daily build (link)
An update on my recent experience with this. Iβve found that I can reliably crash R Studio by creating a model in brms and then carrying out loo() with the option βmoment_match = TRUEβ. It will sometimes crash with other routines such as pp_check(), but that combination is a sure fire way to cause a crash. The interesting thing is that if I store the model in an .RData file, open it up in R (not RStudio) then loo() works fine with moment matching. It would appear to be something to do with how RStudio interacts with R.
Same here. I donβt know why. I have 16 GB RAM and 12 cores but RStudio always crashes when I run a script where I fit two models and then use bayes_factor(). It has no problem if I run it just with R ditching RStudio.
I also have this issue although I noticed they reduce when using options(mc.cores = β¦ ). Moreover, it seems not related to the brms package perse. R studio crashes also when other functions are called upon outside brms (but when brms has been used). Is there already a known cause and possibe solution to this issue? I love the ease of the brms package and feel a bit sad by the R studio issue which somehow removes the efficiency of the particular analysis with brms. Otherwise, any ideas to find out where the issue lies and any diagnostics that could help identify where this issue originates?
For what itβs worth, Iβm experiencing the same problem with loo(, moment_match = TRUE)
. If I remove moment_match = TRUE then loo()
works fine and RStudio does not crash.
The same has been happening to me. Have you been able to find a solution?
Having the same issue when including moment_match = TRUE
. Hopefully someone has found a solution to this.
No solutions. In the meanwhile I started using JAGS, which is great for smaller models, datasets and experimenting. Brms on the other hand has already fixed syntax and model structure. I sometimes 1.) loop through the models 2.) save them in between in a map, 3.) restart automatically after it crashes, 4.) see which models are saved in a the map and 5.) continue from there and repeat this till models have been fit. It is not very βpleasantβ, but perhaps a weird option for some.
(For me at least) using backend = "cmdstanr"
nearly completely eliminated the problem. I canβt remember the last time RStudio crashed because of brms but also only fitted a smaller number of models recently.
Just an update. My box has Ubuntu 20.04.6 LTS here, Ryzen 5950x with 128Gb RAM.
I have large brms objects (900Mb ondisk) and when applying postprocessing convenience functions such as modelsummary::modelsummary(fit), RStudio just crashes and burns. I havenβt tried running it under the terminal, but i will!
George
Yeah, certain functions working on brms
objects also make my RStudio crash.