C++ toolchain problem (Mac OS X Mojave)

[vasishth]

RStan (and brms) suddenly stopped working for me while I was trying to solve some bookdown package update problems.

I followed the instructions here:

RStan Getting Started

For the check to see whether I have build tools, I get:

pkgbuild::has_build_tools(debug = TRUE)
[1] TRUE

Then I tried to run 8schools.

library(rstan)
options(mc.cores = parallel::detectCores())
rstan_options(auto_write = TRUE)

schools_dat <- list(J = 8, 
                    y = c(28,  8, -3,  7, -1,  1, 18, 12),
                    sigma = c(15, 10, 16, 11,  9, 11, 10, 18))

fit <- stan(file = 'schools.stan', data = schools_dat)

Output:

library(rstan)
options(mc.cores = parallel::detectCores())
rstan_options(auto_write = TRUE)
schools_dat ← list(J = 8,

•                 y = c(28,  8, -3,  7, -1,  1, 18, 12),
  

•                 sigma = c(15, 10, 16, 11,  9, 11, 10, 18))
  

fit ← stan(file = ‘schools.stan’, data = schools_dat)
Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! In file included from :1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:82:
In file included from /usr/local/clang7/include/c++/v1/new:91:
In file included from /usr/local/clang7/include/c++/v1/exception:82:
In file included from /usr/local/clang7/include/c++/v1/cstdlib:86:
/usr/local/clang7/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found
#include_next <stdlib.h>
^~~~~~~~~~
1 error generated.
make: *** [filecbe70eb26fe.o] Error 1
In addition: Warning messages:
1: In readLines(file, warn = TRUE) :
incomplete final line found on ‘/Users/shravanvasishth/Downloads/schools.stan’
2: In system(cmd, intern = !verbose) :
running command ‘/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB filecbe70eb26fe.cpp 2> filecbe70eb26fe.cpp.err.txt’ had status 1
Error in sink(type = “output”) : invalid connection```

* Operating System

OS: Mac, 10.14.6 Mojave

* RStan Version

2.19.3

R version 4.0.0 (2020-04-24) -- "Arbor Day"

* Output of `writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))`

> writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))
# clang: start
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
# clang: end

CXX14FLAGS=-O3 -march=native -mtune=native
CXX14FLAGS += -arch x86_64 -ftemplate-depth-256
CPPFLAGS="-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include"
> 

* Output of `devtools::session_info("rstan")`

> devtools::session_info("rstan")
─ Session info ────────────────────────────────────────────────────────────────────────
 setting  value                       
 version  R version 4.0.0 (2020-04-24)
 os       macOS Mojave 10.14.6        
 system   x86_64, darwin17.0          
 ui       RStudio                     
 language (EN)                        
 collate  en_US.UTF-8                 
 ctype    en_US.UTF-8                 
 tz       Europe/Berlin               
 date     2020-05-05                  

─ Packages ────────────────────────────────────────────────────────────────────────────
 package      * version   date       lib source        
 assertthat     0.2.1     2019-03-21 [1] CRAN (R 4.0.0)
 backports      1.1.6     2020-04-05 [1] CRAN (R 4.0.0)
 BH             1.72.0-3  2020-01-08 [1] CRAN (R 4.0.0)
 callr          3.4.3     2020-03-28 [1] CRAN (R 4.0.0)
 checkmate      2.0.0     2020-02-06 [1] CRAN (R 4.0.0)
 cli            2.0.2     2020-02-28 [1] CRAN (R 4.0.0)
 colorspace     1.4-1     2019-03-18 [1] CRAN (R 4.0.0)
 crayon         1.3.4     2017-09-16 [1] CRAN (R 4.0.0)
 desc           1.2.0     2018-05-01 [1] CRAN (R 4.0.0)
 digest         0.6.25    2020-02-23 [1] CRAN (R 4.0.0)
 ellipsis       0.3.0     2019-09-20 [1] CRAN (R 4.0.0)
 evaluate       0.14      2019-05-28 [1] CRAN (R 4.0.0)
 fansi          0.4.1     2020-01-08 [1] CRAN (R 4.0.0)
 farver         2.0.3     2020-01-16 [1] CRAN (R 4.0.0)
 ggplot2      * 3.3.0     2020-03-05 [1] CRAN (R 4.0.0)
 glue           1.4.0     2020-04-03 [1] CRAN (R 4.0.0)
 gridExtra      2.3       2017-09-09 [1] CRAN (R 4.0.0)
 gtable         0.3.0     2019-03-25 [1] CRAN (R 4.0.0)
 inline         0.3.15    2018-05-18 [1] CRAN (R 4.0.0)
 isoband        0.2.1     2020-04-12 [1] CRAN (R 4.0.0)
 labeling       0.3       2014-08-23 [1] CRAN (R 4.0.0)
 lattice        0.20-41   2020-04-02 [1] CRAN (R 4.0.0)
 lifecycle      0.2.0     2020-03-06 [1] CRAN (R 4.0.0)
 loo            2.2.0     2019-12-19 [1] CRAN (R 4.0.0)
 magrittr       1.5       2014-11-22 [1] CRAN (R 4.0.0)
 MASS           7.3-51.5  2019-12-20 [1] CRAN (R 4.0.0)
 Matrix         1.2-18    2019-11-27 [1] CRAN (R 4.0.0)
 matrixStats    0.56.0    2020-03-13 [1] CRAN (R 4.0.0)
 mgcv           1.8-31    2019-11-09 [1] CRAN (R 4.0.0)
 munsell        0.5.0     2018-06-12 [1] CRAN (R 4.0.0)
 nlme           3.1-147   2020-04-13 [1] CRAN (R 4.0.0)
 pillar         1.4.4     2020-05-05 [1] CRAN (R 4.0.0)
 pkgbuild       1.0.7     2020-04-25 [1] CRAN (R 4.0.0)
 pkgconfig      2.0.3     2019-09-22 [1] CRAN (R 4.0.0)
 pkgload        1.0.2     2018-10-29 [1] CRAN (R 4.0.0)
 praise         1.0.0     2015-08-11 [1] CRAN (R 4.0.0)
 prettyunits    1.1.1     2020-01-24 [1] CRAN (R 4.0.0)
 processx       3.4.2     2020-02-09 [1] CRAN (R 4.0.0)
 ps             1.3.2     2020-02-13 [1] CRAN (R 4.0.0)
 R6             2.4.1     2019-11-12 [1] CRAN (R 4.0.0)
 RColorBrewer   1.1-2     2014-12-07 [1] CRAN (R 4.0.0)
 Rcpp           1.0.4.6   2020-04-09 [1] CRAN (R 4.0.0)
 RcppEigen      0.3.3.7.0 2019-11-16 [1] CRAN (R 4.0.0)
 rlang          0.4.5     2020-03-01 [1] CRAN (R 4.0.0)
 rprojroot      1.3-2     2018-01-03 [1] CRAN (R 4.0.0)
 rstan        * 2.19.3    2020-02-11 [1] CRAN (R 4.0.0)
 rstudioapi     0.11      2020-02-07 [1] CRAN (R 4.0.0)
 scales         1.1.0     2019-11-18 [1] CRAN (R 4.0.0)
 StanHeaders  * 2.19.2    2020-02-11 [1] CRAN (R 4.0.0)
 testthat       2.3.2     2020-03-02 [1] CRAN (R 4.0.0)
 tibble         3.0.1     2020-04-20 [1] CRAN (R 4.0.0)
 utf8           1.1.4     2018-05-24 [1] CRAN (R 4.0.0)
 vctrs          0.2.4     2020-03-10 [1] CRAN (R 4.0.0)
 viridisLite    0.3.0     2018-02-01 [1] CRAN (R 4.0.0)
 withr          2.2.0     2020-04-20 [1] CRAN (R 4.0.0)

[1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

[wds15]

with R 4.0 you should not do all that isysroot business and you should just use the stock compiler from Apple. Please lookup the R 4.0 thread on the forum.

[vasishth]

Thanks. I followed this advice from Stephen Martin (is this what you meant?):

"You do not need the macos-rtools anymore, for R 4.0.
I encountered this error in my OSX testing as well.

Rename (or delete) ~/.Renviron and ~/.R/Makevars

That will ‘undo’ the configuration of macos-rtools.
Then test rstan in a new R session."

Now I get:

> fit <- stan(file = 'schools.stan', data = schools_dat)
starting worker pid=3733 on localhost:11245 at 16:51:36.884
starting worker pid=3734 on localhost:11245 at 16:51:36.894
starting worker pid=3732 on localhost:11245 at 16:51:36.899
starting worker pid=3735 on localhost:11245 at 16:51:36.912
Error in unserialize(node$con) : error reading from connection
Calls: <Anonymous> -> slaveLoop -> makeSOCKmaster
Execution halted
Error in unserialize(node$con) : error reading from connection
Calls: <Anonymous> -> slaveLoop -> makeSOCKmaster
Error in unserialize(node$con) : error reading from connection
Calls: <Anonymous> -> slaveLoop -> makeSOCKmaster
Execution halted
Execution halted
Error in unserialize(node$con) : error reading from connection
Calls: <Anonymous> -> slaveLoop -> makeSOCKmaster
Execution halted
Error in makePSOCKcluster(names = spec, ...) : 
  Cluster setup failed. 4 of 4 workers failed to connect.
In addition: Warning message:
In readLines(file, warn = TRUE) :
  incomplete final line found on '/Users/shravanvasishth/Downloads/schools.stan'

[wds15]

another R 4.0 matter; also solved in one of the threads here (sorry, don’t know by heart which one)

[vasishth]

Thanks, I found the discussion here:

Dealing with R 4.0

R version 4.0 was released on April 24. I would recommend that anyone who can update R do so. You have to reinstall all you packages, so do install.packages("rstan", dependencies = TRUE) And there are a few things that you may have to deal with. Windows You should uninstall previous versions of R and RTools. Then do, file.rename(from = "~/.R/Makevars.win", to = "~/.R/Makevars_old") file.remove("~/.R/Makevars") cat("CXX14FLAGS += -mtune=native -march=native -Wno-ignored-attributes -Wno-depre…

I realized from the above that the problem is parallelization in RStudio. I haven’t solved that yet, but I can live with a slow serial compilation for now, at least everything works in principle. Phew, I spent the whole day on this…

[wds15]

I think it was this one:

R 4.0.0 and CRAN MacOS binaries

I also was having issues with multiple chains running in parallel in MacOS using latest R-4.0.0 and RStan. I’ve found out that this is a problem of the package parallel from R-4.0.0 I’ve found this recently issue in RStudio’s GitHub that solved my problem: cl <- parallel::makeCluster(2, setup_strategy = "sequential") If this works, you could add the following to your ~/.Rprofile until this is fixed in R/RStudio: ## WORKAROUND: https://github.com/rstudio/rstudio/issues/6692 ## Revert to 'se…

[vasishth]

I cannot make brms work any more however. I installed brms after getting rstan working (at least through the terminal, not RStudio).

I ran a simple model that until yesterday was working, and I get:

> df_noreading_data <- read_csv("./data/button_press.csv")
Parsed with column specification:
cols(
  rt = col_double(),
  trialn = col_double()
)
> df_noreading_data
# A tibble: 361 x 2
     rt trialn
  <dbl>  <dbl>
1   141      1
2   138      2
3   128      3
4   132      4
5   126      5
# … with 356 more rows
> ggplot(df_noreading_data, aes(rt)) +
+   geom_density() +
+   ggtitle("Button-press data")
> fit_press <- brm(rt ~ 1,
+   data = df_noreading_data,
+   family = gaussian(),
+   prior = c(
+     prior(uniform(0, 60000), class = Intercept),
+     prior(uniform(0, 2000), class = sigma)
+   ),
+   chains = 4,
+   iter = 2000,
+   warmup = 1000
+ )
Compiling the C++ model
Fehler in compileCode(f, code, language = language, verbose = verbose) : 
  Compilation ERROR, function(s)/method(s) not created! In file included from <built-in>:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:82:
In file included from /Users/shravanvasishth/miniconda3/bin/../include/c++/v1/new:89:
In file included from /Users/shravanvasishth/miniconda3/bin/../include/c++/v1/exception:82:
In file included from /Users/shravanvasishth/miniconda3/bin/../include/c++/v1/cstdlib:86:
/Users/shravanvasishth/miniconda3/bin/../include/c++/v1/stdlib.h:94:15: fatal error: 'stdlib.h' file not found
#include_next <stdlib.h>
              ^~~~~~~~~~
1 error generated.
make: *** [file11be6bb4c033.o] Error 1
Zusätzlich: Warnmeldung:
In system(cmd, intern = !verbose) :
  Ausführung von Kommando '/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB file11be6bb4c033.cpp 2> file11be6bb4c033.cpp.err.txt' ergab Status 1
Fehler in sink(type = "output") : ungültige Verbindung

It looks like I do need the C++ libraries etc. that were not needed for rstan?

[bgoodri]

The *conda installations are finicky with Stan and may have their own configuration files. Basically with R 4.0, you want to only use the compiler / libraries / headers / etc. that come with Xcode. Maybe someone else who has RStan installed via miniconda knows how to do that?

[vasishth]

BTW, I didn’t install RStan using miniconda.

[bgoodri]

@vasishth

/Users/shravanvasishth/miniconda3/bin/…/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found

I am not sure why, but it is looking for non-existent files under Users/shravanvasishth/miniconda3/.