Compile issue with Rstan on Macbook Pro M1

Good day

I have struggling to get my Rstan working again on my M1 Mac, perhaps some updates messed with it.

I have followed the instructions on the Getting Started Page from scratch, but when running the example I get the following error.

Error in compileCode(f, code, language = language, verbose = verbose) : 
                                          ^/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:835:49: note: expanded from macro '_LIBCPP_DEPRECATED'#      define _LIBCPP_DEPRECATED __attribute__((deprecated))                                                ^97 warnings and 1 error generated.make: *** [file158a97e3a7a53.o] Error 1
Error in sink(type = "output") : invalid connection

Session Info:

sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 14.0

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rstan_2.32.3       StanHeaders_2.32.2 ggplot2_3.4.2     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11        compiler_4.2.0     pillar_1.8.1       QuickJSR_1.0.7     prettyunits_1.1.1 
 [6] tools_4.2.0        pkgbuild_1.3.1     jsonlite_1.8.0     lifecycle_1.0.3    tibble_3.1.8      
[11] gtable_0.3.1       pkgconfig_2.0.3    rlang_1.1.1        cli_3.6.1          rstudioapi_0.14   
[16] curl_4.3.2         parallel_4.2.0     loo_2.5.1          gridExtra_2.3      withr_2.5.0       
[21] dplyr_1.0.10       generics_0.1.3     vctrs_0.6.2        tidyselect_1.1.2   stats4_4.2.0      
[26] grid_4.2.0         inline_0.3.19      glue_1.6.2         R6_2.5.1           processx_3.7.0    
[31] fansi_1.0.3        purrr_0.3.4        callr_3.7.2        magrittr_2.0.3     codetools_0.2-18  
[36] matrixStats_0.62.0 scales_1.2.1       ps_1.7.1           colorspace_2.0-3   V8_4.2.1          
[41] utf8_1.2.2         RcppParallel_5.1.5 munsell_0.5.0      crayon_1.5.1      

Anything obvious I can try, since I"'m really having a hard time with compilers on the Mac.

Thanks

Can you provide which version of the OS and Xcode?

Sure, sorry I should have done that from the start:

macOS Sonoma Version 14.0 (23A344)
xcode-select version 2399

I’m not sure I needed the full installation of Xcode? I didn’t gather from the quickstart of Rstan that that is a requirement, but I think you are on to something. How do I know which compiler Rstan is going to use.

I installed XCode Version 15.0.1, but I got the same error.

I know back a version I had to remove all the rstan stuff and reinstall it for it to work. I am a version back on the M1 OS and Xcode at the moment. Once my models finishing running I’ll upgrade and see what’s going on.

This thread might help Error in sink(type = "output") : invalid connection on Mac OS

Thanks @Ara_Winter, I will look into this, but unfortunately, so far it looks like most of the libraries like gfortran are for Mac OS 13 and not 14, so there might be a version issue. For security reasons, I will probably not be allowed to downgrade. I will repeat the steps in the linked thread again and see if I have any luck.

Just upgraded. So before I upgraded, I removed rstan, brms, and other packages that use rstan. Not sure that’s necessary. Quit out of R and ran the upgrade.

Clean reinstall of rstan and brms through the package manager in RStudio (really need to bail out of RStudio).

So now I am running on OS14 and Xcode version 2403 on the M1 chips. Both brms models and stan models compile and run.

Unfortunately, these steps didn’t work for me. The only difference is that I didn’t remove all the packages before upgrading, because I have already the latest OS. I guess I will repeat those steps sometime soon. In the meantime I got cmdstan working, which is strange, because if there was a compiler issue, I would assume that cmdstan would also not work.

Oh weird. Yeah last time I had errors due to the mac os and xcode neither rstan, brms, or cmdstan worked. Let dig around and see if I can recreate your error message on my machine.

Is your include directory still intact after the update. I know on some upgrades in the past mac os will wipe that out.

Thanks for all the help. I also have an unsolved issue. Also originally here: Can't compile code using RStan

Here’s the full error now. I reinstalled everything including CommandLineTools. I really don’t know what the problem is.

> brm(mpg~1,mtcars)
Compiling Stan program...
make cmd is
  make -f '/Library/Frameworks/R.framework/Resources/etc/Makeconf' -f '/Library/Frameworks/R.framework/Resources/share/make/shlib.mk' -f '/Users/justinsavoie/.R/Makevars' CXX='$(CXX17) $(CXX17STD)' CXXFLAGS='$(CXX17FLAGS)' CXXPICFLAGS='$(CXX17PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX17LDFLAGS)' SHLIB_LD='$(SHLIB_CXX17LD)' SHLIB='file18ab72d229a7.so' OBJECTS='file18ab72d229a7.o'

make would use
clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/Rcpp/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppEigen/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppEigen/include/unsupported"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/BH/include" -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/include/src/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppParallel/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/rstan/include" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DUSE_STANC3 -DSTRICT_R_HEADERS  -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION  -D_HAS_AUTO_PTR_ETC=0  -include '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/include/stan/math/prim/fun/Eigen.hpp'  -D_REENTRANT -DRCPP_PARALLEL_USE_TBB=1   -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include    -fPIC  -O3 -fPIC -c file18ab72d229a7.cpp -o file18ab72d229a7.o
if test  "zfile18ab72d229a7.o" != "z"; then \
	  echo clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L"/Library/Frameworks/R.framework/Resources/lib" -L/opt/R/arm64/lib -o file18ab72d229a7.so file18ab72d229a7.o  '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/rstan/lib//libStanServices.a' -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/lib/' -lStanHeaders -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppParallel/lib/' -ltbb   -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation; \
	  clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L"/Library/Frameworks/R.framework/Resources/lib" -L/opt/R/arm64/lib -o file18ab72d229a7.so file18ab72d229a7.o  '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/rstan/lib//libStanServices.a' -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/lib/' -lStanHeaders -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppParallel/lib/' -ltbb   -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation; \
	fi
Error in compileCode(f, code, language = language, verbose = verbose) : 
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/cstring:66:5: error: <cstring> tried including <string.h> but didn't find libc++'s <string.h> header.           This usually means that your header search paths are not configured properly.           The header search paths should contain the C++ Standard Library headers before           any C Standard Library, and you are probably using compiler flags that make that           not be the case.#   error <cstring> tried including <string.h> but didn't find libc++'s <string.h> header. \    ^fatal error: too many errors emitted, stopping now [-ferror-limit=]20 errors generated.make: *** [file18ab72d229a7.o] Error 1
Error in sink(type = "output") : invalid connection

I have Sonoma 14.0 (23A344).
I have xcode-select version 2399.

Additionally:

g++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I found this here: ⚙ D131441 [libc++] Diagnose when header search paths are set up incorrectly which says "For example, people will manually add -isystem /usr/include, which ends up messing up the layering of header search paths required by libc++ (because the C Standard Library now appears before libc++ in
the search paths). " but I still don’t know how to fix it.

At the end of the day it’s probably this: “The header search paths should contain the C++ Standard Library headers before any C Standard Library, and you are probably using compiler flags that make that not be the case.” but again; not sure.

There are some folks here terminal - Unable to find include file on newly updated macOS(C/C++) - Ask Different wrangling with a similar issue.

Let me see what versions I have of g++, apple clang.

Hi @Ara_Winter it appears that it is still intact. Unfortunately that Stan typically has these compiler issues, especially after updates, which makes it so hard to get going. Still a mystery that my cmdstan works.

Okay I got it working, but not sure what fixed it. I can tell you what I did. I simply updated all my R packages in Rstudio, by clicking on the “update” button in the library tab. I suspect that some libraries related to Stan headers or other C++ libraries (like RcppParallel) maybe helped to resolve the issue.

Here are some Stan-related libraries:

Here is my latest SessionInfo:

R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 14.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rstan_2.32.3       StanHeaders_2.32.2 ggplot2_3.4.4     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11        pillar_1.9.0       compiler_4.2.0     QuickJSR_1.0.7     prettyunits_1.2.0  tools_4.2.0       
 [7] pkgbuild_1.4.2     jsonlite_1.8.7     lifecycle_1.0.4    tibble_3.2.1       gtable_0.3.4       pkgconfig_2.0.3   
[13] rlang_1.1.1        cli_3.6.1          rstudioapi_0.15.0  curl_5.1.0         parallel_4.2.0     loo_2.6.0         
[19] gridExtra_2.3      withr_2.5.2        dplyr_1.1.3        generics_0.1.3     vctrs_0.6.4        stats4_4.2.0      
[25] grid_4.2.0         tidyselect_1.2.0   glue_1.6.2         inline_0.3.19      R6_2.5.1           processx_3.8.2    
[31] fansi_1.0.5        callr_3.7.3        magrittr_2.0.3     codetools_0.2-19   matrixStats_0.62.0 scales_1.2.1      
[37] ps_1.7.5           colorspace_2.1-0   V8_4.4.0           utf8_1.2.4         RcppParallel_5.1.7 munsell_0.5.0     
[43] crayon_1.5.2
1 Like

Glad it’s working. Maybe we should get something together about what do when the mac os updates breaks things.

Also fixed the problem.

What I did:I removed Command Line Tools. I updated Sonoma to 14.1.1. I deleted completely Homebrew (this was radical and I’m not sure it was needed, but I didn’t really need Homebrew). I installed the latest regular (non-beta) Xcode. I had tried before with CLT only; that didn’t work. CLT 15.1 beta didn’t work for me. Actually all CLT didn’t work for me.

So, update sonoma to latest, update Xcode to latest.

This: Gcc 13.1.0 macOS Sonoma not compiling Ada or any languages on Intel or M1 Macs - Stack Overflow

explains that there’s a problem with Sonoma and Xcode. I suspect this was the issue.

1 Like

I wish I could be more specific as to what worked. All I can say is that I uninstalled all things related to Rstan, then followed the steps in the Getting Started Page multiple times, downloaded Xcode and made sure that all R-libraries were up to date. It sounds a bit like “have tried restarting your computer” since these are just all the basic steps.