R 4.0.0 and CRAN MacOS binaries

Gfortran for Stan, really?

No, that is for other stuff

I installed R 4.0.0 RC on Debian bullseye (11). I used this in .R/Makevars

CXX14FLAGS=-O3 -march=native -mtune=native -fPIC
CXX14=g++

I used install.packages("rstan", type = "source"), which installed rstan 2.19.3 and StanHeaders 2.19.2.

The 8schools example ran without a problem.

2 Likes

What version of g++ do you have? I got a student’s Mac to work using clang++ from Xcode with some effort.

g++ (Debian 9.3.0-10) 9.3.0

Hi This is my first post to this group and I am a novice. I am working with Imperial’s covid model 3 on an Imac and have just updated to R 4.0.0
I had to move to an old machine not running Catalina to run and experiment with their earlier models because of the reasons that have been discussed between Stan and Catalina. However this machine is MUCH more powerful so need to move my research now onto this machine. So Rstan now works but I have the same problem by bgoodri above- I cannot get 4 cores to sample just 1 works. Is there anything I can do to overcome this please??? this is the error message from R

[1] “First non-zero cases is on day 64, and 30 days before 10 deaths is day 53”
[1] “Portugal has 64 days of data”
[1] “First non-zero cases is on day 60, and 30 days before 10 deaths is day 46”
[1] “Netherlands has 71 days of data”
starting worker pid=45437 on localhost:11120 at 20:50:04.418
starting worker pid=45434 on localhost:11120 at 20:50:04.418
starting worker pid=45436 on localhost:11120 at 20:50:04.419
starting worker pid=45435 on localhost:11120 at 20:50:04.419
Error in unserialize(node$con) : error reading from connection
Calls: -> slaveLoop -> makeSOCKmaster
Error in unserialize(node$con) : error reading from connection
Calls: -> slaveLoop -> makeSOCKmaster
Execution halted
Execution halted
Error in unserialize(node$con) : error reading from connection
Calls: -> slaveLoop -> makeSOCKmaster
Error in unserialize(node$con) : error reading from connection
Calls: -> slaveLoop -> makeSOCKmaster
Execution halted
Execution halted
Error in makePSOCKcluster(names = spec, …) :
Cluster setup failed. 4 of 4 workers failed to connect.


and this is my set up

sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.4

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base

other attached packages:
[1] rstanarm_2.19.3 xts_0.12-0 zoo_1.8-7 rstantools_2.0.0 Rcpp_1.0.4.6 abind_1.4-5 ggpubr_0.2.5 magrittr_1.5
[9] bayesplot_1.7.1 scales_1.1.0 stringr_1.4.0 optparse_1.6.6 EnvStats_2.3.1 tidyr_1.0.2 dplyr_0.8.5 gdata_2.18.0
[17] data.table_1.12.8 rstan_2.19.3 ggplot2_3.3.0 StanHeaders_2.19.2 lubridate_1.7.8 httr_1.4.1 readr_1.3.1

loaded via a namespace (and not attached):
[1] nlme_3.1-147 matrixStats_0.56.0 threejs_0.3.3 tools_4.0.0 R6_2.4.1 DT_0.13 colorspace_1.4-1 withr_2.2.0
[9] tidyselect_1.0.0 gridExtra_2.3 prettyunits_1.1.1 processx_3.4.2 curl_4.3 compiler_4.0.0 cli_2.0.2 shinyjs_1.1
[17] colourpicker_1.0 dygraphs_1.1.1.6 ggridges_0.5.2 callr_3.4.3 digest_0.6.25 minqa_1.2.4 base64enc_0.1-3 pkgconfig_2.0.3
[25] htmltools_0.4.0 lme4_1.1-23 fastmap_1.0.1 htmlwidgets_1.5.1 rlang_0.4.5 shiny_1.4.0.2 generics_0.0.2 crosstalk_1.1.0.1
[33] gtools_3.8.2 inline_0.3.15 loo_2.2.0 Matrix_1.2-18 munsell_0.5.0 fansi_0.4.1 lifecycle_0.2.0 stringi_1.4.6
[41] MASS_7.3-51.5 pkgbuild_1.0.6 plyr_1.8.6 grid_4.0.0 promises_1.1.0 crayon_1.3.4 miniUI_0.1.1.1 lattice_0.20-41
[49] splines_4.0.0 hms_0.5.3 ps_1.3.2 pillar_1.4.3 igraph_1.2.5 boot_1.3-24 markdown_1.1 ggsignif_0.6.0
[57] shinystan_2.5.0 reshape2_1.4.4 codetools_0.2-16 stats4_4.0.0 glue_1.4.0 packrat_0.5.0 nloptr_1.2.2.1 vctrs_0.2.4
[65] httpuv_1.5.2 gtable_0.3.0 getopt_1.20.3 purrr_0.3.3 assertthat_0.2.1 mime_0.9 xtable_1.8-4 later_1.0.0
[73] survival_3.1-12 rsconnect_0.8.16 tibble_3.0.1 shinythemes_1.1.2 statmod_1.4.34 ellipsis_0.3.0

many thanks for your help
Mark

This is an RStudio bug

It should run with multiple cores if either you knit, run R from the Terminal, or tell RStudio to run your script as a separate job. If RStudio can’t fix this soon, we’ll put a workaround for it into rstan.

1 Like

Many thanks Ben,
Mark

Or the R GUI for Mac should work

Sorry Ben not having much luck-- is this
not an OpenMP problem in data.table - get this now on Mac GUI–
but stopped I think for a different reason…

  > library(data.table)

  data.table 1.12.8 using 1 threads (see ?getDTthreads).  Latest

news: r-datatable.com

I don’t know if the data.table package has any incompatibilities with the new R stuff.

I also was having issues with multiple chains running in parallel in MacOS using latest R-4.0.0 and RStan.

I’ve found out that this is a problem of the package parallel from R-4.0.0

I’ve found this recently issue in RStudio’s GitHub that solved my problem:


cl <- parallel::makeCluster(2, setup_strategy = "sequential")
If this works, you could add the following to your ~/.Rprofile until this is fixed in R/RStudio:

## WORKAROUND: https://github.com/rstudio/rstudio/issues/6692
## Revert to 'sequential' setup of PSOCK cluster in RStudio Console on macOS and R 4.0.0
if (Sys.getenv("RSTUDIO") == "1" && !nzchar(Sys.getenv("RSTUDIO_TERM")) && 
    Sys.info()["sysname"] == "Darwin" && getRversion() == "4.0.0") {
  parallel:::setDefaultClusterOptions(setup_strategy = "sequential")
}

Hope this helps clarifying the issues surrounding RStan, MacOS and R-4.0.0

10 Likes

Thank you very much- you also solved my problem as well- after three weeks of pain with various Rstan, R and Rstudio problems I can get back to work!

Thanks for sharing this with the community! This is exactly why I started this thread in order for everyone to share their experience and solutions to the various hiccups. It’s work to sort it out and more work to share that, but many will benefit.

1 Like

Worked for me too. Thanks!

If I try to fit a Stan model using CTRL+Enter in RStudio I get an error, but If I run it as a local job inside the RStudio IDE it works ok. That can also work as a solution.

Running on MacOs Catalina, R 4.0.0, with no .Makevars or .Rprofile.

1 Like

I have been having the same issues running RStan and the ~/.Rprofile still gave me an error message:
Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! In file included from :1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:82:
In file included from /usr/local/clang7/include/c++/v1/new:91:
In file included from /usr/local/clang7/include/c++/v1/exception:82:
In file included from /usr/local/clang7/include/c++/v1/cstdlib:86:
/usr/local/clang7/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found
#include_next <stdlib.h>
^~~~~~~~~~
1 error generated.
make: *** [file5e3542fec36.o] Error 1
In addition: Warning message:
In system(cmd, intern = !verbose) :
running command ‘/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB file5e3542fec36.cpp 2> file5e3542fec36.cpp.err.txt’ had status 1
Error in sink(type = “output”) : invalid connection

I tried bgautijonsson’s run as local job solution, but it also gave me an error:

Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.19.3, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)
Running /Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB foo.c
Trying to compile a simple C file
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/unsupported" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/BH/include" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/src/" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/rstan/include" -DEIGEN_NO_DEBUG -D_REENTRANT -DBOOST_DISABLE_ASSERTS -DBOOST_PENDING_INTEGER_LOG2_HPP -include stan/math/prim/mat/fun/Eigen.hpp -I/usr/local/include -fPIC -Wall -g -O2 -c foo.c -o foo.o
In file included from :1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:88:
/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/src/Core/util/Macros.h:613:1: error: unknown type name ‘namespace’
namespace Eigen {
^
/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/src/Core/util/Macros.h:613:16: error: expected ‘;’ after top level declarator
namespace Eigen {
^
;
In file included from :1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:96:10: fatal error: ‘complex’ file not found
#include
^~~~~~~~~
3 errors generated.
make: *** [foo.o] Error 1
Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! In file included from :1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:82:
In file included from /usr/local/clang7/include/c++/v1/new:91:
In file included from /usr/local/clang7/include/c++/v1/exception:82:
In file included from /usr/local/clang7/include/c++/v1/cstdlib:86:
/usr/local/clang7/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found
#include_next <stdlib.h>
^~~~~~~~~~
1 error generated.
make: *** [file69d1fe691b4.o] Error 1
Calls: sourceWithProgress … cxxfunctionplus -> -> cxxfunction -> compileCode
In addition: Warning message:
In system(cmd, intern = !verbose) :
running command ‘/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB file69d1fe691b4.cpp 2> file69d1fe691b4.cpp.err.txt’ had status 1
Error in sink(type = “output”) : invalid connection
Calls: sourceWithProgress … eval -> stan -> stan_model -> cxxfunctionplus -> sink
Execution halted

Is there an additional step I should take for this to be fixed?

I believe the above is your problem. Remove anything in your paths where you point to clang7.

Thanks for this! Just a quick comment: R 4.0.1 has recently been released so getRversion() == "4.0.0" is now outdated - I would recommend to simply remove this part of the expression.

2 Likes

Thanks so much! Had the same problem, fix worked for me too with R version 4.0.1