Problems running brms models with cores > 4

  • Operating System: OS High Sierra
  • brms Version: 2.11.1

Hi there,

I love brms, and have been using it weekly for a couple of years now. Recently, I have had a sporadic issue when I run a model with e.g. 4 chains across 4 cores, on my fairly good Mac desktop (32GB RAM). The problem is that the model runs fine, but the execution never completes after the chains have all finished. That is, I get the notifications from Stan in R’s console that all 4 chains have ended, but the code never completes and returns to the > prompt, and I need to force-quit R. If I run the same model with cores = 1, everything works fine. This occurs when running even basic models using all the brms defaults except for cores > 1. I enclose my sessionInfo and the contents of ~/.R/Makevars (I have a poor understanding of what is in there - I only edit it when trying to load odd packages and programs, and this could be the issue). Grateful for any ideas, as the multicore functionality is really handy, and until recently (I’m not sure when) it has always “just worked”, even for far bigger models (in terms of RAM, complexity, and/or number of iterations) than I am currently working on.

All the best!

Luke

My makevars:

This file’s location is ~/.R/Makevars

Default variables (no omp support):

CXX=clang++

CC=clang

CC="/usr/local/opt/llvm/bin/clang"
CXX="/usr/local/opt/llvm/bin/clang++"
CXX14FLAGS=-O3 -march=native -mtune=native
CXX14="/usr/local/opt/llvm/bin/clang++"

My session info:

R version 3.6.2 (2019-12-12)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] kableExtra_1.1.0 gridExtra_2.3 brms_2.11.1 Rcpp_1.0.3 tidybayes_2.0.1 ggbeeswarm_0.6.0 forcats_0.4.0
[8] stringr_1.4.0 dplyr_0.8.4 purrr_0.3.3 readr_1.3.1 tidyr_1.0.2 tibble_2.1.3 ggplot2_3.2.1
[15] tidyverse_1.3.0

loaded via a namespace (and not attached):
[1] colorspace_1.4-1 ggridges_0.5.2 rsconnect_0.8.16 markdown_1.1 base64enc_0.1-3 fs_1.3.1
[7] rstudioapi_0.11 rstan_2.19.2 svUnit_0.7-12 DT_0.12 fansi_0.4.1 mvtnorm_1.0-12
[13] lubridate_1.7.4 xml2_1.2.2 bridgesampling_0.8-1 knitr_1.28 shinythemes_1.1.2 bayesplot_1.7.1
[19] jsonlite_1.6.1 packrat_0.5.0 broom_0.5.4 dbplyr_1.4.2 shiny_1.4.0 compiler_3.6.2
[25] httr_1.4.1 backports_1.1.5 assertthat_0.2.1 Matrix_1.2-18 fastmap_1.0.1 lazyeval_0.2.2
[31] cli_2.0.1 later_1.0.0 htmltools_0.4.0 prettyunits_1.1.1 tools_3.6.2 igraph_1.2.4.2
[37] coda_0.19-3 gtable_0.3.0 glue_1.3.1 reshape2_1.4.3 cellranger_1.1.0 vctrs_0.2.2
[43] nlme_3.1-144 crosstalk_1.0.0 xfun_0.12 ps_1.3.0 rvest_0.3.5 mime_0.9
[49] miniUI_0.1.1.1 lifecycle_0.1.0 gtools_3.8.1 zoo_1.8-7 scales_1.1.0 colourpicker_1.0
[55] hms_0.5.3 promises_1.1.0 Brobdingnag_1.2-6 parallel_3.6.2 inline_0.3.15 shinystan_2.5.0
[61] loo_2.2.0 StanHeaders_2.21.0-1 stringi_1.4.5 dygraphs_1.1.1.6 pkgbuild_1.0.6 rlang_0.4.4
[67] pkgconfig_2.0.3 matrixStats_0.55.0 evaluate_0.14 lattice_0.20-38 rstantools_2.0.0 htmlwidgets_1.5.1
[73] tidyselect_1.0.0 processx_3.4.2 plyr_1.8.5 magrittr_1.5 R6_2.4.1 generics_0.0.2
[79] DBI_1.1.0 pillar_1.4.3 haven_2.2.0 withr_2.1.2 xts_0.12-0 abind_1.4-5
[85] modelr_0.1.5 crayon_1.3.4 arrayhelpers_1.1-0 rmarkdown_2.1 grid_3.6.2 readxl_1.3.1
[91] callr_3.4.1 threejs_0.3.3 webshot_0.5.2 reprex_0.3.0 digest_0.6.23 xtable_1.8-4
[97] httpuv_1.5.2 stats4_3.6.2 munsell_0.5.0 viridisLite_0.3.0 beeswarm_0.2.3 vipor_0.4.5
[103] shinyjs_1.1

Hi Luke!

Are you also running into problems using rstan or rstanarm?

I’m definitely not the right person to solve this issue, but let’s see if we can get the right people in here to make this work for you again.

Cheers,
Max

Hi Max,

I haven’t tried those recently - I can’t write Stan code, and I like the extra complexity offered by brms over rstanarm. Also I just noticed a typo in the title of the post - it should say cores > 1. I can give rstanarm a try next week to help isolate the problem, that’s a good idea.

Cheers!

Luke, I experienced the same the other day and just killed everything. Yesterday before leaving work the same thing happened (brms GitHub d/l yesterday). I left it and when I came this morning I had the prompt waiting for me. Could be a regression in the code perhaps, but then I guess @paul.buerkner might know if anyone has reported this before?

1 Like

@paul.buerkner, this happened again now. I had a model that took 24h to sample. All chains finished almost 24 hours ago now, and I still haven’t gotten the prompt back. Is there anything I can do to: 1. getting it unstuck without dropping my samples, 2. trying to figure out what is causing this so it won’t happen again?

I honestly don’t know since we don’t know yet at which level of the code it fails. I would need a reproducible example but I understand this will be very hard to get.

1 Like

I am not sure if brms uses mcapply or if @torkar experienced this in RStudio but this might be related to this rstudio issue https://github.com/rstudio/rstudio/issues/2597

1 Like

Yeah there seems to be a general consencus that forked parallelism in Rstudio is not stable - see this issue: https://github.com/HenrikBengtsson/future/issues/299

Indeed in the future.lapply package it gives this warning when you try to use forked processing:

I don’t know any more than this but had some instability with mclapply recently that was solved after switching to future.lapply with the ‘multisession’ mode of parallel processing.

If forking is indeed causing the issue, then using the future option with brms and setting up the plan correctly should make this problem go away.

Hvala lepa, @rok_cesnovar! I believe this could definitely be it, since I use RStudio. Bloody hard to reproduce on my side, as Paul said…

1 Like