Compile issue with Rstan on Macbook Pro M1

Tikavdm · October 30, 2023, 9:24am

Good day

I have struggling to get my Rstan working again on my M1 Mac, perhaps some updates messed with it.

I have followed the instructions on the Getting Started Page from scratch, but when running the example I get the following error.

Error in compileCode(f, code, language = language, verbose = verbose) : 
                                          ^/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:835:49: note: expanded from macro '_LIBCPP_DEPRECATED'#      define _LIBCPP_DEPRECATED __attribute__((deprecated))                                                ^97 warnings and 1 error generated.make: *** [file158a97e3a7a53.o] Error 1
Error in sink(type = "output") : invalid connection

Session Info:

sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 14.0

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rstan_2.32.3       StanHeaders_2.32.2 ggplot2_3.4.2     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11        compiler_4.2.0     pillar_1.8.1       QuickJSR_1.0.7     prettyunits_1.1.1 
 [6] tools_4.2.0        pkgbuild_1.3.1     jsonlite_1.8.0     lifecycle_1.0.3    tibble_3.1.8      
[11] gtable_0.3.1       pkgconfig_2.0.3    rlang_1.1.1        cli_3.6.1          rstudioapi_0.14   
[16] curl_4.3.2         parallel_4.2.0     loo_2.5.1          gridExtra_2.3      withr_2.5.0       
[21] dplyr_1.0.10       generics_0.1.3     vctrs_0.6.2        tidyselect_1.1.2   stats4_4.2.0      
[26] grid_4.2.0         inline_0.3.19      glue_1.6.2         R6_2.5.1           processx_3.7.0    
[31] fansi_1.0.3        purrr_0.3.4        callr_3.7.2        magrittr_2.0.3     codetools_0.2-18  
[36] matrixStats_0.62.0 scales_1.2.1       ps_1.7.1           colorspace_2.0-3   V8_4.2.1          
[41] utf8_1.2.2         RcppParallel_5.1.5 munsell_0.5.0      crayon_1.5.1

Anything obvious I can try, since I"'m really having a hard time with compilers on the Mac.

Thanks

Ara_Winter · October 30, 2023, 11:09pm

Can you provide which version of the OS and Xcode?

Tikavdm · October 31, 2023, 9:14am

Sure, sorry I should have done that from the start:

macOS Sonoma Version 14.0 (23A344)
xcode-select version 2399

I’m not sure I needed the full installation of Xcode? I didn’t gather from the quickstart of Rstan that that is a requirement, but I think you are on to something. How do I know which compiler Rstan is going to use.

Tikavdm · October 31, 2023, 10:40am

I installed XCode Version 15.0.1, but I got the same error.

Ara_Winter · October 31, 2023, 1:20pm

I know back a version I had to remove all the rstan stuff and reinstall it for it to work. I am a version back on the M1 OS and Xcode at the moment. Once my models finishing running I’ll upgrade and see what’s going on.

This thread might help Error in sink(type = "output") : invalid connection on Mac OS

Tikavdm · November 2, 2023, 9:21am

Thanks @Ara_Winter, I will look into this, but unfortunately, so far it looks like most of the libraries like gfortran are for Mac OS 13 and not 14, so there might be a version issue. For security reasons, I will probably not be allowed to downgrade. I will repeat the steps in the linked thread again and see if I have any luck.

Ara_Winter · November 4, 2023, 2:21pm

Just upgraded. So before I upgraded, I removed rstan, brms, and other packages that use rstan. Not sure that’s necessary. Quit out of R and ran the upgrade.

Clean reinstall of rstan and brms through the package manager in RStudio (really need to bail out of RStudio).

So now I am running on OS14 and Xcode version 2403 on the M1 chips. Both brms models and stan models compile and run.

Tikavdm · November 9, 2023, 12:04pm

Unfortunately, these steps didn’t work for me. The only difference is that I didn’t remove all the packages before upgrading, because I have already the latest OS. I guess I will repeat those steps sometime soon. In the meantime I got cmdstan working, which is strange, because if there was a compiler issue, I would assume that cmdstan would also not work.

Ara_Winter · November 9, 2023, 2:16pm

Oh weird. Yeah last time I had errors due to the mac os and xcode neither rstan, brms, or cmdstan worked. Let dig around and see if I can recreate your error message on my machine.

Ara_Winter · November 9, 2023, 2:20pm

Is your include directory still intact after the update. I know on some upgrades in the past mac os will wipe that out.

Justin_Savoie · November 14, 2023, 2:37am

Thanks for all the help. I also have an unsolved issue. Also originally here: Can't compile code using RStan

Here’s the full error now. I reinstalled everything including CommandLineTools. I really don’t know what the problem is.

> brm(mpg~1,mtcars)
Compiling Stan program...
make cmd is
  make -f '/Library/Frameworks/R.framework/Resources/etc/Makeconf' -f '/Library/Frameworks/R.framework/Resources/share/make/shlib.mk' -f '/Users/justinsavoie/.R/Makevars' CXX='$(CXX17) $(CXX17STD)' CXXFLAGS='$(CXX17FLAGS)' CXXPICFLAGS='$(CXX17PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX17LDFLAGS)' SHLIB_LD='$(SHLIB_CXX17LD)' SHLIB='file18ab72d229a7.so' OBJECTS='file18ab72d229a7.o'

make would use
clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/Rcpp/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppEigen/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppEigen/include/unsupported"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/BH/include" -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/include/src/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppParallel/include/"  -I"/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/rstan/include" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DUSE_STANC3 -DSTRICT_R_HEADERS  -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION  -D_HAS_AUTO_PTR_ETC=0  -include '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/include/stan/math/prim/fun/Eigen.hpp'  -D_REENTRANT -DRCPP_PARALLEL_USE_TBB=1   -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include    -fPIC  -O3 -fPIC -c file18ab72d229a7.cpp -o file18ab72d229a7.o
if test  "zfile18ab72d229a7.o" != "z"; then \
	  echo clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L"/Library/Frameworks/R.framework/Resources/lib" -L/opt/R/arm64/lib -o file18ab72d229a7.so file18ab72d229a7.o  '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/rstan/lib//libStanServices.a' -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/lib/' -lStanHeaders -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppParallel/lib/' -ltbb   -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation; \
	  clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L"/Library/Frameworks/R.framework/Resources/lib" -L/opt/R/arm64/lib -o file18ab72d229a7.so file18ab72d229a7.o  '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/rstan/lib//libStanServices.a' -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/StanHeaders/lib/' -lStanHeaders -L'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/RcppParallel/lib/' -ltbb   -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation; \
	fi
Error in compileCode(f, code, language = language, verbose = verbose) : 
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/cstring:66:5: error: <cstring> tried including <string.h> but didn't find libc++'s <string.h> header.           This usually means that your header search paths are not configured properly.           The header search paths should contain the C++ Standard Library headers before           any C Standard Library, and you are probably using compiler flags that make that           not be the case.#   error <cstring> tried including <string.h> but didn't find libc++'s <string.h> header. \    ^fatal error: too many errors emitted, stopping now [-ferror-limit=]20 errors generated.make: *** [file18ab72d229a7.o] Error 1
Error in sink(type = "output") : invalid connection

I have Sonoma 14.0 (23A344).
I have xcode-select version 2399.

Additionally:

g++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I found this here: ⚙ D131441 [libc++] Diagnose when header search paths are set up incorrectly which says "For example, people will manually add -isystem /usr/include, which ends up messing up the layering of header search paths required by libc++ (because the C Standard Library now appears before libc++ in
the search paths). " but I still don’t know how to fix it.

At the end of the day it’s probably this: “The header search paths should contain the C++ Standard Library headers before any C Standard Library, and you are probably using compiler flags that make that not be the case.” but again; not sure.

Ara_Winter · November 14, 2023, 2:09pm

There are some folks here terminal - Unable to find include file on newly updated macOS(C/C++) - Ask Different wrangling with a similar issue.

Let me see what versions I have of g++, apple clang.

Tikavdm · November 16, 2023, 7:33am

Hi @Ara_Winter it appears that it is still intact. Unfortunately that Stan typically has these compiler issues, especially after updates, which makes it so hard to get going. Still a mystery that my cmdstan works.

Tikavdm · November 16, 2023, 8:48am

Okay I got it working, but not sure what fixed it. I can tell you what I did. I simply updated all my R packages in Rstudio, by clicking on the “update” button in the library tab. I suspect that some libraries related to Stan headers or other C++ libraries (like RcppParallel) maybe helped to resolve the issue.

Here are some Stan-related libraries:

Here is my latest SessionInfo:

R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 14.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rstan_2.32.3       StanHeaders_2.32.2 ggplot2_3.4.4     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11        pillar_1.9.0       compiler_4.2.0     QuickJSR_1.0.7     prettyunits_1.2.0  tools_4.2.0       
 [7] pkgbuild_1.4.2     jsonlite_1.8.7     lifecycle_1.0.4    tibble_3.2.1       gtable_0.3.4       pkgconfig_2.0.3   
[13] rlang_1.1.1        cli_3.6.1          rstudioapi_0.15.0  curl_5.1.0         parallel_4.2.0     loo_2.6.0         
[19] gridExtra_2.3      withr_2.5.2        dplyr_1.1.3        generics_0.1.3     vctrs_0.6.4        stats4_4.2.0      
[25] grid_4.2.0         tidyselect_1.2.0   glue_1.6.2         inline_0.3.19      R6_2.5.1           processx_3.8.2    
[31] fansi_1.0.5        callr_3.7.3        magrittr_2.0.3     codetools_0.2-19   matrixStats_0.62.0 scales_1.2.1      
[37] ps_1.7.5           colorspace_2.1-0   V8_4.4.0           utf8_1.2.4         RcppParallel_5.1.7 munsell_0.5.0     
[43] crayon_1.5.2

Ara_Winter · November 16, 2023, 1:55pm

Glad it’s working. Maybe we should get something together about what do when the mac os updates breaks things.

Justin_Savoie · November 17, 2023, 2:19pm

Also fixed the problem.

What I did:I removed Command Line Tools. I updated Sonoma to 14.1.1. I deleted completely Homebrew (this was radical and I’m not sure it was needed, but I didn’t really need Homebrew). I installed the latest regular (non-beta) Xcode. I had tried before with CLT only; that didn’t work. CLT 15.1 beta didn’t work for me. Actually all CLT didn’t work for me.

So, update sonoma to latest, update Xcode to latest.

This: Gcc 13.1.0 macOS Sonoma not compiling Ada or any languages on Intel or M1 Macs - Stack Overflow

explains that there’s a problem with Sonoma and Xcode. I suspect this was the issue.

Tikavdm · November 22, 2023, 1:58pm

I wish I could be more specific as to what worked. All I can say is that I uninstalled all things related to Rstan, then followed the steps in the Getting Started Page multiple times, downloaded Xcode and made sure that all R-libraries were up to date. It sounds a bit like “have tried restarting your computer” since these are just all the basic steps.

Topic		Replies	Views
Trouble with initial use of rstan Interfaces rstan	2	628	November 11, 2019
MacBook M1 users - reports of problems with Big Sur 11.3 Developers	2	1186	May 13, 2021
Cannot compile Rstan Code on Mac M1 and cannot instan cmdstan General rstan , mac , compiler	1	34	December 12, 2024
Error in sink(type = "output") : invalid connection after using mrctools to ensure everything is in the place RStan brms	1	553	April 7, 2023
Error in compileCode(), Compilation ERROR, Warning message: In system(cmd, intern = !verbose) , Error in sink(type = "output") RStan	4	571	May 12, 2020

Compile issue with Rstan on Macbook Pro M1

Related topics