RStudio crashes working with brms

any particular method for which it fails?

I can’t say for sure, but I think the trend in post-processing is that functions calling to some form of posterior predictives vs posterior samples result in the crash. Most readily I find that pp_check() and add_criterion() are the biggest culprits, but those are also the post-processing things I call the most frequently so that might just be selection bias. I know that doesn’t explain the crashes that sometimes happen after sampling or right during compiling, but those happen very rarely for me and almost always it’s because I took the shortcut of not deleting an old fit before running the model again in the same session.

In my mind, the posterior predictions being a cause for Rstudio crashes makes sense if it is a memory commitment issue and Rstudio having a low threshold for crashing. It’s been a while now, but I recall seeing discussions on CrossValidated discussing the seemingly arbitrary memory allotment for Windows + Rstudio. My guess (which I really make with a grain of salt because I’m talking out the wrong end of me on these kinds of topics) is that generating those predictions and manipulating sometimes very large arrays probably produces some kind of spike in computing needs that trips something in Rstudio. Possibly, setting mc.cores to less than the total available helps avert Rstudio from believing that its going to over commit the computer’s resources (?)

1 Like

Yeah. It might indeed be a problem with the default of using more than 1 core on windows in some post-processing functions (Change default `cores` for `posterior_predict` on Windows Β· Issue #1129 Β· paul-buerkner/brms Β· GitHub). I will change that later on today and let you know once its done.

Just fixed it in the github version of brms. If the crashes are because of accidental use of multiple cores in post-processing functions on windows, it should no longer occur.

2 Likes

I’m having a similar issue using Ubuntu 20.04. I can fit models with brms, but then using post-processing functions creates crashes. After updating with the github version I could get the first plot (but not the second), and now RStudio crashes always when executing the first plot function.

library(brms)
#> Loading required package: Rcpp
#> Loading 'brms' package (version 2.15.5). Useful instructions
#> can be found by typing help('brms'). A more detailed introduction
#> to the package is available through vignette('brms_overview').
#> 
#> Attaching package: 'brms'
#> The following object is masked from 'package:stats':
#> 
#>     ar
options(mc.cores = parallel::detectCores())

fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient), 
            data = epilepsy, family = poisson())
#> Compiling Stan program...
#> Start sampling
summary(fit1) 
#>  Family: poisson 
#>   Links: mu = log 
#> Formula: count ~ zAge + zBase * Trt + (1 | patient) 
#>    Data: epilepsy (Number of observations: 236) 
#> Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup samples = 4000
#> 
#> Group-Level Effects: 
#> ~patient (Number of levels: 59) 
#>               Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> sd(Intercept)     0.59      0.07     0.46     0.74 1.00     1010     1634
#> 
#> Population-Level Effects: 
#>            Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept      1.77      0.11     1.54     1.99 1.01      755     1643
#> zAge           0.10      0.09    -0.08     0.26 1.00      821     1188
#> zBase          0.70      0.12     0.46     0.93 1.00      767     1434
#> Trt1          -0.27      0.17    -0.59     0.06 1.00      812     1692
#> zBase:Trt1     0.06      0.16    -0.28     0.38 1.00      917     1492
#> 
#> Samples were drawn using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

plot(conditional_effects(fit1, effects = "zBase:Trt"))
plot(fit1, pars = c("Trt", "zBase")) 
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.0 (2021-05-18)
#>  os       Ubuntu 20.04.2 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Europe/Berlin               
#>  date     2021-05-26                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package        * version date       lib source                             
#>  abind            1.4-5   2016-07-21 [1] CRAN (R 4.1.0)                     
#>  assertthat       0.2.1   2019-03-21 [1] CRAN (R 4.1.0)                     
#>  backports        1.2.1   2020-12-09 [1] CRAN (R 4.1.0)                     
#>  base64enc        0.1-3   2015-07-28 [1] CRAN (R 4.1.0)                     
#>  bayesplot        1.8.0   2021-01-10 [1] CRAN (R 4.1.0)                     
#>  boot             1.3-28  2021-05-03 [4] CRAN (R 4.0.5)                     
#>  bridgesampling   1.1-2   2021-04-16 [1] CRAN (R 4.1.0)                     
#>  brms           * 2.15.5  2021-05-26 [1] Github (paul-buerkner/brms@fdd776c)
#>  Brobdingnag      1.2-6   2018-08-13 [1] CRAN (R 4.1.0)                     
#>  callr            3.7.0   2021-04-20 [1] CRAN (R 4.1.0)                     
#>  cli              2.5.0   2021-04-26 [1] CRAN (R 4.1.0)                     
#>  coda             0.19-4  2020-09-30 [1] CRAN (R 4.1.0)                     
#>  codetools        0.2-18  2020-11-04 [4] CRAN (R 4.0.3)                     
#>  colorspace       2.0-1   2021-05-04 [1] CRAN (R 4.1.0)                     
#>  colourpicker     1.1.0   2020-09-14 [1] CRAN (R 4.1.0)                     
#>  crayon           1.4.1   2021-02-08 [1] CRAN (R 4.1.0)                     
#>  crosstalk        1.1.1   2021-01-12 [1] CRAN (R 4.1.0)                     
#>  curl             4.3.1   2021-04-30 [1] CRAN (R 4.1.0)                     
#>  DBI              1.1.1   2021-01-15 [1] CRAN (R 4.1.0)                     
#>  digest           0.6.27  2020-10-24 [1] CRAN (R 4.1.0)                     
#>  dplyr            1.0.6   2021-05-05 [1] CRAN (R 4.1.0)                     
#>  DT               0.18    2021-04-14 [1] CRAN (R 4.1.0)                     
#>  dygraphs         1.1.1.6 2018-07-11 [1] CRAN (R 4.1.0)                     
#>  ellipsis         0.3.2   2021-04-29 [1] CRAN (R 4.1.0)                     
#>  emmeans          1.6.0   2021-04-24 [1] CRAN (R 4.1.0)                     
#>  estimability     1.3     2018-02-11 [1] CRAN (R 4.1.0)                     
#>  evaluate         0.14    2019-05-28 [1] CRAN (R 4.1.0)                     
#>  fansi            0.5.0   2021-05-25 [1] CRAN (R 4.1.0)                     
#>  fastmap          1.1.0   2021-01-25 [1] CRAN (R 4.1.0)                     
#>  fs               1.5.0   2020-07-31 [1] CRAN (R 4.1.0)                     
#>  gamm4            0.2-6   2020-04-03 [1] CRAN (R 4.1.0)                     
#>  generics         0.1.0   2020-10-31 [1] CRAN (R 4.1.0)                     
#>  ggplot2          3.3.3   2020-12-30 [1] CRAN (R 4.1.0)                     
#>  ggridges         0.5.3   2021-01-08 [1] CRAN (R 4.1.0)                     
#>  glue             1.4.2   2020-08-27 [1] CRAN (R 4.1.0)                     
#>  gridExtra        2.3     2017-09-09 [1] CRAN (R 4.1.0)                     
#>  gtable           0.3.0   2019-03-25 [1] CRAN (R 4.1.0)                     
#>  gtools           3.8.2   2020-03-31 [1] CRAN (R 4.1.0)                     
#>  highr            0.9     2021-04-16 [1] CRAN (R 4.1.0)                     
#>  htmltools        0.5.1.1 2021-01-22 [1] CRAN (R 4.1.0)                     
#>  htmlwidgets      1.5.3   2020-12-10 [1] CRAN (R 4.1.0)                     
#>  httpuv           1.6.1   2021-05-07 [1] CRAN (R 4.1.0)                     
#>  igraph           1.2.6   2020-10-06 [1] CRAN (R 4.1.0)                     
#>  inline           0.3.18  2021-05-18 [1] CRAN (R 4.1.0)                     
#>  jsonlite         1.7.2   2020-12-09 [1] CRAN (R 4.1.0)                     
#>  knitr            1.33    2021-04-24 [1] CRAN (R 4.1.0)                     
#>  later            1.2.0   2021-04-23 [1] CRAN (R 4.1.0)                     
#>  lattice          0.20-44 2021-05-02 [4] CRAN (R 4.1.0)                     
#>  lifecycle        1.0.0   2021-02-15 [1] CRAN (R 4.1.0)                     
#>  lme4             1.1-27  2021-05-15 [1] CRAN (R 4.1.0)                     
#>  loo              2.4.1   2020-12-09 [1] CRAN (R 4.1.0)                     
#>  magrittr         2.0.1   2020-11-17 [1] CRAN (R 4.1.0)                     
#>  markdown         1.1     2019-08-07 [1] CRAN (R 4.1.0)                     
#>  MASS             7.3-54  2021-05-03 [4] CRAN (R 4.0.5)                     
#>  Matrix           1.3-3   2021-05-04 [4] CRAN (R 4.0.5)                     
#>  matrixStats      0.58.0  2021-01-29 [1] CRAN (R 4.1.0)                     
#>  mgcv             1.8-35  2021-04-18 [4] CRAN (R 4.0.5)                     
#>  mime             0.10    2021-02-13 [1] CRAN (R 4.1.0)                     
#>  miniUI           0.1.1.1 2018-05-18 [1] CRAN (R 4.1.0)                     
#>  minqa            1.2.4   2014-10-09 [1] CRAN (R 4.1.0)                     
#>  multcomp         1.4-17  2021-04-29 [1] CRAN (R 4.1.0)                     
#>  munsell          0.5.0   2018-06-12 [1] CRAN (R 4.1.0)                     
#>  mvtnorm          1.1-1   2020-06-09 [1] CRAN (R 4.1.0)                     
#>  nlme             3.1-152 2021-02-04 [4] CRAN (R 4.0.3)                     
#>  nloptr           1.2.2.2 2020-07-02 [1] CRAN (R 4.1.0)                     
#>  pillar           1.6.1   2021-05-16 [1] CRAN (R 4.1.0)                     
#>  pkgbuild         1.2.0   2020-12-15 [1] CRAN (R 4.1.0)                     
#>  pkgconfig        2.0.3   2019-09-22 [1] CRAN (R 4.1.0)                     
#>  plyr             1.8.6   2020-03-03 [1] CRAN (R 4.1.0)                     
#>  prettyunits      1.1.1   2020-01-24 [1] CRAN (R 4.1.0)                     
#>  processx         3.5.2   2021-04-30 [1] CRAN (R 4.1.0)                     
#>  projpred         2.0.2   2020-10-28 [1] CRAN (R 4.1.0)                     
#>  promises         1.2.0.1 2021-02-11 [1] CRAN (R 4.1.0)                     
#>  ps               1.6.0   2021-02-28 [1] CRAN (R 4.1.0)                     
#>  purrr            0.3.4   2020-04-17 [1] CRAN (R 4.1.0)                     
#>  R6               2.5.0   2020-10-28 [1] CRAN (R 4.1.0)                     
#>  Rcpp           * 1.0.6   2021-01-15 [1] CRAN (R 4.1.0)                     
#>  RcppParallel     5.1.4   2021-05-04 [1] CRAN (R 4.1.0)                     
#>  reprex           2.0.0   2021-04-02 [1] CRAN (R 4.1.0)                     
#>  reshape2         1.4.4   2020-04-09 [1] CRAN (R 4.1.0)                     
#>  rlang            0.4.11  2021-04-30 [1] CRAN (R 4.1.0)                     
#>  rmarkdown        2.8     2021-05-07 [1] CRAN (R 4.1.0)                     
#>  rsconnect        0.8.18  2021-05-24 [1] CRAN (R 4.1.0)                     
#>  rstan            2.26.1  2021-05-26 [1] local                              
#>  rstantools       2.1.1   2020-07-06 [1] CRAN (R 4.1.0)                     
#>  rstudioapi       0.13    2020-11-12 [1] CRAN (R 4.1.0)                     
#>  sandwich         3.0-1   2021-05-18 [1] CRAN (R 4.1.0)                     
#>  scales           1.1.1   2020-05-11 [1] CRAN (R 4.1.0)                     
#>  sessioninfo      1.1.1   2018-11-05 [1] CRAN (R 4.1.0)                     
#>  shiny            1.6.0   2021-01-25 [1] CRAN (R 4.1.0)                     
#>  shinyjs          2.0.0   2020-09-09 [1] CRAN (R 4.1.0)                     
#>  shinystan        2.5.0   2018-05-01 [1] CRAN (R 4.1.0)                     
#>  shinythemes      1.2.0   2021-01-25 [1] CRAN (R 4.1.0)                     
#>  StanHeaders      2.26.1  2021-05-26 [1] local                              
#>  stringi          1.6.2   2021-05-17 [1] CRAN (R 4.1.0)                     
#>  stringr          1.4.0   2019-02-10 [1] CRAN (R 4.1.0)                     
#>  survival         3.2-11  2021-04-26 [4] CRAN (R 4.0.5)                     
#>  TH.data          1.0-10  2019-01-21 [1] CRAN (R 4.1.0)                     
#>  threejs          0.3.3   2020-01-21 [1] CRAN (R 4.1.0)                     
#>  tibble           3.1.2   2021-05-16 [1] CRAN (R 4.1.0)                     
#>  tidyselect       1.1.1   2021-04-30 [1] CRAN (R 4.1.0)                     
#>  utf8             1.2.1   2021-03-12 [1] CRAN (R 4.1.0)                     
#>  V8               3.4.2   2021-05-01 [1] CRAN (R 4.1.0)                     
#>  vctrs            0.3.8   2021-04-29 [1] CRAN (R 4.1.0)                     
#>  withr            2.4.2   2021-04-18 [1] CRAN (R 4.1.0)                     
#>  xfun             0.23    2021-05-15 [1] CRAN (R 4.1.0)                     
#>  xtable           1.8-4   2019-04-21 [1] CRAN (R 4.1.0)                     
#>  xts              0.12.1  2020-09-09 [1] CRAN (R 4.1.0)                     
#>  yaml             2.2.1   2020-02-01 [1] CRAN (R 4.1.0)                     
#>  zoo              1.8-9   2021-03-09 [1] CRAN (R 4.1.0)                     
#> 
#> [1] /home/cmoya/R/x86_64-pc-linux-gnu-library/4.1
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library

It seems to be a problem on the RStudio side. The code runs if directly executed in the terminal. It seems to be fixed by using the latest daily build (link)

1 Like

An update on my recent experience with this. I’ve found that I can reliably crash R Studio by creating a model in brms and then carrying out loo() with the option β€˜moment_match = TRUE’. It will sometimes crash with other routines such as pp_check(), but that combination is a sure fire way to cause a crash. The interesting thing is that if I store the model in an .RData file, open it up in R (not RStudio) then loo() works fine with moment matching. It would appear to be something to do with how RStudio interacts with R.

1 Like

Same here. I don’t know why. I have 16 GB RAM and 12 cores but RStudio always crashes when I run a script where I fit two models and then use bayes_factor(). It has no problem if I run it just with R ditching RStudio.

I also have this issue although I noticed they reduce when using options(mc.cores = … ). Moreover, it seems not related to the brms package perse. R studio crashes also when other functions are called upon outside brms (but when brms has been used). Is there already a known cause and possibe solution to this issue? I love the ease of the brms package and feel a bit sad by the R studio issue which somehow removes the efficiency of the particular analysis with brms. Otherwise, any ideas to find out where the issue lies and any diagnostics that could help identify where this issue originates?

1 Like

For what it’s worth, I’m experiencing the same problem with loo(, moment_match = TRUE). If I remove moment_match = TRUE then loo() works fine and RStudio does not crash.

2 Likes

The same has been happening to me. Have you been able to find a solution?

1 Like

Having the same issue when including moment_match = TRUE. Hopefully someone has found a solution to this.

2 Likes

No solutions. In the meanwhile I started using JAGS, which is great for smaller models, datasets and experimenting. Brms on the other hand has already fixed syntax and model structure. I sometimes 1.) loop through the models 2.) save them in between in a map, 3.) restart automatically after it crashes, 4.) see which models are saved in a the map and 5.) continue from there and repeat this till models have been fit. It is not very β€œpleasant”, but perhaps a weird option for some.

1 Like

(For me at least) using backend = "cmdstanr" nearly completely eliminated the problem. I can’t remember the last time RStudio crashed because of brms but also only fitted a smaller number of models recently.

1 Like

Just an update. My box has Ubuntu 20.04.6 LTS here, Ryzen 5950x with 128Gb RAM.

I have large brms objects (900Mb ondisk) and when applying postprocessing convenience functions such as modelsummary::modelsummary(fit), RStudio just crashes and burns. I haven’t tried running it under the terminal, but i will!

George

1 Like

Yeah, certain functions working on brms objects also make my RStudio crash.