Then I don’t know how this
Warning message:
In find.package(package, lib.loc, verbose = verbose) :
package ‘base’ found more than once, using the first from
“/Library/Frameworks/R.framework/Resources/library/base”,
“/Library/Frameworks/R.framework/Versions/4.0/Resources/library/base”
happens. Maybe something doesn’t know that something else was deleted?
I don’t know how calling some ggplot2 stuff on a matrix of draws that has already been extracted from the stanfit object would cause problems. Can you try
rm(m1_fit)
gc(TRUE)
m1_samples %>%
ggplot(aes(x = name, y = value)) +
geom_violin()
I don’t know what could be going on here
√ @dbarneche R.framework: ls -lG
/Library/Frameworks/R.framework
total 0
drwxrwxr-x 8 root admin 256 19 Sep 14:36 .
drwxr-xr-x 5 root wheel 160 19 Sep 14:36 ..
lrwxr-xr-x 1 root admin 24 19 Sep 14:36 Headers -> Versions/Current/Headers
lrwxr-xr-x 1 root admin 30 19 Sep 14:36 Libraries -> Versions/Current/Resources/lib
lrwxr-xr-x 1 root admin 31 19 Sep 14:36 PrivateHeaders -> Versions/Current/PrivateHeaders
lrwxr-xr-x 1 root admin 18 19 Sep 14:36 R -> Versions/Current/R
lrwxr-xr-x 1 root admin 26 19 Sep 14:36 Resources -> Versions/Current/Resources
drwxrwxr-x 4 root admin 128 19 Sep 14:36 Versions
It looks like R automatically creates links pointing /R.framework/Resources/ to /R.framework/Versions/Current/Resources/ . Yesterday I removed R, XCode command line tools, reinstalled Catalina, installed XCode and R from scratch, and got to the same crash malloc errors. In any case, it seems that the crashes shouldn’t be related to this warning msg?
You could try
remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan")
It might be related to the tbbmalloc_proxy stuff, but this would be the first time we have seen that.
@bgoodri I think this did the trick! Thanks so much! After running 10 different models none of them returned the malloc messages before or after the model fit, and no crashes observed after that.
The warning message about finding two base paths with the verbose rstan example still exists, so I’m not sure what is going on there.
This is the output I’m getting now:
> set.seed(1)
> x <- rnorm(100, 10)
> df <- data.frame(x = x,
+ y = 2 + x * 0.5 + rnorm(100, 0.1, 0.1))
>
> fit <- brm(y ~ x, data = df)
Compiling Stan program...
'config' variable 'CPP' is deprecated
clang -mmacosx-version-min=10.13 -E
Start sampling
SAMPLING FOR MODEL '83d86b6377f3277965dbdafcc8ab07c9' NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 2.3e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.23 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
Chain 1: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 1: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 1: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 1: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 1: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 1: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 1: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 1: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 1: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 1: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 1: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 1: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 1:
Chain 1: Elapsed Time: 0.022834 seconds (Warm-up)
Chain 1: 0.021255 seconds (Sampling)
Chain 1: 0.044089 seconds (Total)
Chain 1:
SAMPLING FOR MODEL '83d86b6377f3277965dbdafcc8ab07c9' NOW (CHAIN 2).
Chain 2:
Chain 2: Gradient evaluation took 9e-06 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.09 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2:
Chain 2:
Chain 2: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 2: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 2: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 2: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 2: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 2: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 2: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 2: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 2: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 2: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 2: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 2: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 2:
Chain 2: Elapsed Time: 0.022564 seconds (Warm-up)
Chain 2: 0.0216 seconds (Sampling)
Chain 2: 0.044164 seconds (Total)
Chain 2:
SAMPLING FOR MODEL '83d86b6377f3277965dbdafcc8ab07c9' NOW (CHAIN 3).
Chain 3:
Chain 3: Gradient evaluation took 8e-06 seconds
Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.08 seconds.
Chain 3: Adjust your expectations accordingly!
Chain 3:
Chain 3:
Chain 3: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 3: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 3: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 3: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 3: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 3: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 3: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 3: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 3: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 3: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 3: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 3: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 3:
Chain 3: Elapsed Time: 0.021711 seconds (Warm-up)
Chain 3: 0.0225 seconds (Sampling)
Chain 3: 0.044211 seconds (Total)
Chain 3:
SAMPLING FOR MODEL '83d86b6377f3277965dbdafcc8ab07c9' NOW (CHAIN 4).
Chain 4:
Chain 4: Gradient evaluation took 7e-06 seconds
Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0.07 seconds.
Chain 4: Adjust your expectations accordingly!
Chain 4:
Chain 4:
Chain 4: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 4: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 4: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 4: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 4: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 4: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 4: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 4: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 4: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 4: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 4: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 4: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 4:
Chain 4: Elapsed Time: 0.021362 seconds (Warm-up)
Chain 4: 0.02275 seconds (Sampling)
Chain 4: 0.044112 seconds (Total)
Chain 4:
Warning message:
In system(paste(CPP, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
error in running command
session info:
> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.6
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] brms_2.13.5 Rcpp_1.0.5
loaded via a namespace (and not attached):
[1] Brobdingnag_1.2-6 jsonlite_1.7.1 gtools_3.8.2 StanHeaders_2.21.0-6
[5] RcppParallel_5.0.2 threejs_0.3.3 shiny_1.5.0 assertthat_0.2.1
[9] stats4_4.0.2 backports_1.1.10 pillar_1.4.6 lattice_0.20-41
[13] glue_1.4.2 digest_0.6.25 promises_1.1.1 colorspace_1.4-1
[17] htmltools_0.5.0 httpuv_1.5.4 Matrix_1.2-18 plyr_1.8.6
[21] dygraphs_1.1.1.6 pkgconfig_2.0.3 rstan_2.21.3 purrr_0.3.4
[25] xtable_1.8-4 mvtnorm_1.1-1 scales_1.1.1 processx_3.4.4
[29] later_1.1.0.1 emmeans_1.5.0 tibble_3.0.3 bayesplot_1.7.2
[33] generics_0.0.2 ggplot2_3.3.2 ellipsis_0.3.1 DT_0.15
[37] withr_2.2.0 shinyjs_2.0.0 cli_2.0.2 magrittr_1.5
[41] crayon_1.3.4 mime_0.9 estimability_1.3 ps_1.3.4
[45] fansi_0.4.1 nlme_3.1-148 xts_0.12.1 pkgbuild_1.1.0
[49] colourpicker_1.1.0 rsconnect_0.8.16 tools_4.0.2 loo_2.3.1
[53] prettyunits_1.1.1 lifecycle_0.2.0 matrixStats_0.56.0 stringr_1.4.0
[57] V8_3.2.0 munsell_0.5.0 callr_3.4.4 compiler_4.0.2
[61] rlang_0.4.7 grid_4.0.2 ggridges_0.5.2 htmlwidgets_1.5.1
[65] crosstalk_1.1.0.1 igraph_1.2.5 miniUI_0.1.1.1 base64enc_0.1-3
[69] codetools_0.2-16 gtable_0.3.0 inline_0.3.16 abind_1.4-5
[73] curl_4.3 markdown_1.1 reshape2_1.4.4 R6_2.4.1
[77] gridExtra_2.3 rstantools_2.1.1 zoo_1.8-8 bridgesampling_1.0-0
[81] dplyr_1.0.2 fastmap_1.0.1 shinystan_2.5.0 shinythemes_1.1.2
[85] stringi_1.5.3 parallel_4.0.2 vctrs_0.3.4 tidyselect_1.1.0
[89] coda_0.19-3
I updated to Catalina v10.15.6 yesterday and that seems to have fixed the problem (that isn’t why I updated, but seems to have worked anyways). Perhaps this problem has been some weird interaction between RStudio, R, and Catalina. Thanks for everyone’s help, though!
@wds Installing from github (which does not use tbbmalloc_proxy on any Mac) seems to have fixed the malloc error @dbarneche was having. Does that give us any clue as to why tbbmalloc_proxy was nondeterministically crashing R CMD check?
For me too, installing the development version of RStan from Github solved the problem (with javascript=F; the new compiler doesn’t work). Before that, I tried the newest binary and source versions from CRAN (2.21.1, and then 2.21.2 from source, probably).
Hi there, first post. I’m also encountering an issue like this. The model samples fine, but right when it gets to the end of sampling and is in the process of saving to an R object, boom, session aborted. It doesn’t happen every time, but it happens often enough to be incredibly frustrating, especially as the model I am working with takes several hours minimum to run (I know that is nothing compared to what some go through). It will also occasionally happen when sampling from the posterior using the rethinking function “link”. It’s hard to be certain, but it seems to happen more often when overwriting an existing R object.
Like our OP, I am using the rstan via the rethinking package. Newest versions of R, Rstudio, rstan and all associated packages as of this morning (with the exception of withr, which is 2.2.0), and I’m using Windows. I tried to install the development version of rstan as recommended in this thread, but that did not work and generated the following output:
Downloading GitHub repo stan-dev/rstan@develop
/usr/bin/tar: stan-dev-rstan-2f29681/StanHeaders/inst/include/libsundials: Cannot create symlink to ‘mathlib/lib/sundials_4.1.0/’: No such file or directory
/usr/bin/tar: stan-dev-rstan-2f29681/StanHeaders/inst/include/src: Cannot create symlink to ‘upstream/src/’: No such file or directory
/usr/bin/tar: stan-dev-rstan-2f29681/StanHeaders/inst/include/stan: Cannot create symlink to ‘mathlib/stan/’: No such file or directory
/usr/bin/tar: Exiting with failure status due to previous errors
External tar failed with--force-local, trying without
/usr/bin/tar: stan-dev-rstan-2f29681/StanHeaders/inst/include/libsundials: Cannot create symlink to ‘mathlib/lib/sundials_4.1.0/’: No such file or directory
/usr/bin/tar: stan-dev-rstan-2f29681/StanHeaders/inst/include/src: Cannot create symlink to ‘upstream/src/’: No such file or directory
/usr/bin/tar: stan-dev-rstan-2f29681/StanHeaders/inst/include/stan: Cannot create symlink to ‘mathlib/stan/’: No such file or directory
/usr/bin/tar: Exiting with failure status due to previous errors
“C:\PROGRA~1\Git\cmd\git.exe” clone --depth 1 --no-hardlinks --recurse-submodules --branch develop GitHub - stan-dev/stan: Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details. C:\Users\KEVIN_~1\AppData\Local\Temp\RtmpSMtcsb\remotes1cdc4d4d802/stan-dev-rstan-2f29681/rstan/rstan/…/…/StanHeaders/inst/include/upstream
Error: Failed to install ‘rstan’ from GitHub:
Command failed (128)
In addition: Warning messages:
1: In utils::untar(tarfile, …) :
‘tar.exe -xf “C:\Users\KEVIN_~1\AppData\Local\Temp\RtmpSMtcsb\file1cdc496c6b5a.tar.gz” -C “C:/Users/KEVIN_~1/AppData/Local/Temp/RtmpSMtcsb/remotes1cdc4d4d802”’ returned error code 2
2: In system(full, intern = TRUE, ignore.stderr = quiet) :
running command ‘“C:\PROGRA~1\Git\cmd\git.exe” clone --depth 1 --no-hardlinks --recurse-submodules --branch develop GitHub - stan-dev/stan: Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details. C:\Users\KEVIN_~1\AppData\Local\Temp\RtmpSMtcsb\remotes1cdc4d4d802/stan-dev-rstan-2f29681/rstan/rstan/…/…/StanHeaders/inst/include/upstream’ had status 128
Any advice here? Seems like others had luck fixing this problem with rstan development version, so would be nice if that did the trick for me too.
Thanks
I have ended up using cmdstanr and converting to an rstan object for plotting etc., but even then R from RStudio sometimes crashes with the first plot after fitting. That is relatively rare and manageable by saving the fit object as an RDS file, just in case. This instability may be unrelated to crashes at the end of rstan model fitting.
I don’t have a case available for testing, but a contributing factor to crashes may be that there are both tidyr::extract and rstan::extract, and that at least I currently have in the global namespace the tidyr version.
I have a very similar problem. Running a fairly straightforward hierarchical model. Sampling goes fine, extracting the posterior goes fine, but it aborts on the plotting of the means.
Error message:
R(2872,0x1119c9dc0) malloc: Attempted to register zone more than once: 0x106387110
R(2872,0x1119c9dc0) malloc: Non-aligned pointer 0x10cb5b940 being freed (2)
R(2872,0x1119c9dc0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6
Are there any updates on a fix? Happy to share reproducible example and system info. I have tried running from RStudio, from R within Terminal on my Mac and from RStudioServer on a HPC.