Dealing with Catalina II

Thanks for all the quick replies!

ls /usr/local gives

Cellar bin clang6 clang8 f-secure include man remotedesktop texlive
Homebrew clang4 clang7 etc gfortran lib opt share var

See anything wrong?

Interestingly, I restarted R and removed the saved Stan model (.rds file) again. This time when running it tells me Building R package from source requires installation of additional build tools. Do you want ot install the additional tools now?

Since I had removed the .rds files before posting here, this change must have come from updating the Makevars file with @bgoodri 's post. Any ideas what I’m missing now?
xcode-select --install tells me the command line tools are already installed.

Something is off on your system. I’m not sure what though.

Could you give me:

  1. A screenshot of the startup text / RStudio IDE panel
  2. Details about your R installation via:
utils::sessionInfo()

Apparently I was a bit sloppy. When removing the flags from @bgoodri’s post from Makevars, I still have R ask me to install additional build tools as soon as I start compiling a Stan model. Not sure why this happens, as it didn’t happen before.

  1. Not a screenshot but hopefully this works for you:

R version 3.6.1 (2019-07-05) – “Action of the Toes”
Copyright © 2019 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.6.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type ‘license()’ or ‘licence()’ for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type ‘contributors()’ for more information and
‘citation()’ on how to cite R or R packages in publications.

Type ‘demo()’ for some demos, ‘help()’ for on-line help, or
‘help.start()’ for an HTML browser interface to help.
Type ‘q()’ to quit R.

  1. This is the output:
    R version 3.6.1 (2019-07-05)
    Platform: x86_64-apple-darwin15.6.0 (64-bit)
    Running under: macOS Catalina 10.15.1

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached):
[1] compiler_3.6.1 tools_3.6.1 packrat_0.5.0

Obviously, if you have something better to do on a weekend than look at this I respect that. This is not super urgent :-)

Oh, one more thing. I just tried the Rcpp throw_exception thing again and now I get the following error:

/Users/mauritsm/.R/Makevars:7: *** Recursive variable `PATH’ references itself (eventually). Stop.
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Rcpp/include" -I"/private/var/folders/87/4lknlk2d2c39ph1ty1xtmwnr0000gn/T/RtmpefxTCL/sourceCpp-x86_64-apple-darwin15.6.0-1.0.3" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c file12a479465fc1.cpp -o file12a479465fc1.o
Error in Rcpp::sourceCpp(code = “\n#include <Rcpp.h>\nusing namespace Rcpp; \n\n// [[Rcpp::export]]\nint throw_exception() { \n std::stringstream errmsg; errmsg << “this is an exception”;\n throw std::domain_error(errmsg.str()); \n return 0;\n}\n”) :
Error 1 occurred building shared library.

Sys.getenv('PATH') gives:
[1] “/usr/local/clang7/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:export PATH=/usr/local/clang6/bin:$PATH:/Library/TeX/texbin:/opt/X11/bin”

Hope this is useful information - I’m definitely not an expert at this…

This is what you need to sort out, I think. That’s not a rstan problem at all. You PATH definition is messed up apparently.

1 Like

Here is my steps to solve the problem:

  1. Update Xcode to the newest version.
  2. In terminal: xcode-select —install (install the Xcode CLI)
  3. Install the r-macos-rtools by @coatless
  4. In Finder: Go → Go to Folders… → ~/.R/Makevars and modify the Makevars file as @wds15’s suggestion :

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX11LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib

  1. (Maybe optional)In R: upgrade to R 3.6.1, remove rstan and StanHeaders (maybe optional) and reinstall rstan:

remove.packages(“rstan”)
remove.packages(“StanHeaders”)
file.remove(“.RData”)

restart R

install.packages(“rstan”, dependencies = TRUE)

1 Like

I did some Googling and was able to find what was wrong with my setup. Then I restarted my computer and reran the model from before and all chains are sampling now. Thanks @bgoodri, @coatless and @wds15 for your help with this!

Same for me, if I run @coatless as-is then I get sampling errors every now and then. But if I change .R/Makevars to

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX11LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib

then it works.

3 Likes

Ok not a specific Catalina question here but as I’m thinking to buy a new macbook pro - going forward will Stan GPU acceleration continue to work on Mac now that day have deprecated OpenCl in favour of Metal2?

I hope this question isn’t out of place here I didnt’ think it merited a new thread.

The question is really out of place here… but to my understanding is that its very uncertain that GPU support will be there on Mac in the future given that OpenCL support is dropped. The hope is to find some adaptor software which handles this though. There are some discussions scattered on the forum.

1 Like

I think Sebastian mostly answered your question. Just to add a bit more context, though this might be a bit offtopic.

The only currently supported compute language on recent MacOSes is Metal. The only other non-fully-experimental way of doing GPU computing on a Mac is using Vulkan Compute and then translating that to metal with MoltenVK.

There is no OpenCL support as well as no CUDA support. Well NVIDIA GPUs dont even have drivers for Macs. There is an experimental translator from OpenCL to Vulkan which could then be tranlslate to Metal. But that all seems very bizzare.

The bigger isssue is that Metal has no double-precision floating point support which makes is really not suitable for Stan.

The only real hope for Stan GPU on a Mac is that Apple adds back suport for OpenCL or that they add support for OpenCL-Next or SYCL or even CUDA in the future.

1 Like

Hello! I am trying to run rstanarm on my MacBook Pro (Catalina). I appear to be running into the same issues listed in forums previously, but can’t make any of the suggested fixes fix my problem. I have downloaded and successfully installed the latest installer, updated R, RStudio, and my Mac (running 10.15.1), and restarted, re-downloaded, and rebooted as suggested. When running stan_lmer, I still come up with the same error message after the first chain:

[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)”
error occurred during calling the sampler; sampling not done

And then at the close of several failed chains:
[[4]]
Stan model ‘continuous’ does not contain samples.
Error in check_stanfit(stanfit) :
Invalid stanfit object produced please report bug
In addition: Warning messages:
1: In .local(object, …) :
some chains had errors; consider specifying chains = 1 to debug
2: In .local(object, …) :
some chains had errors; consider specifying chains = 1 to debug
Error in dimnamesGets(x, value) :
invalid dimnames given for “dgCMatrix” object

Any help you can offer would be greatly appreciated. My code and example code works just fine on my PC. Thanks so much for your time. -Amy

For rstanarm, at the moment, you have to reinstall it from source with

install.packages("rstanarm", type = "source")

Thanks Ben- unfortunately, both rstan and rstanarm are indeed installed from source. I just re-installed again just to be sure, but receive the same error.

Can you try installing rstanarm from source again after adding

CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

to ~/.R/Makevars ?

Hey guys - this thread has become very confusing for those of us dealing with this issue for the first time. There are a lot of recommended code snippets, but its very hard to tell what makes them different from each other, or whether any of them are worth trying at this point, etc.

Since this is a widespread issue, I want to suggest that it would be helpful for the community if someone knowledgeable put up a page, perhaps on the git, explaining the situation, and describing the current state of potentially effective workarounds, if there are any.

(BTW - the same issue arises with llvm9 installed by homebrew.)

6 Likes

The one at the top post is the recommended one when compiling your own Stan programs. We still don’t know what to do about packages that come with Stan models because the way they are linked on CRAN does not work when the user has Catalina.

@bgoodri The advice at the top of the thread (use the coatless installer to point the C include flags to XCode) did not fix building Stan models for me. (Possibly because XCode has been updated since R was compiled? Perhaps it would make sense to experiment with different XCode versions?) I think several other people have given the same comment.

I wasn’t aware that the latest Xcode update broke the solution to the breakage of the previous Xcode update. @coatless ? @ssp3nc3r ?

Just to be clear - I’m not saying the new XCode broke the fix. I’m just saying the fix didn’t work for me. The new XCode version was a guess about a possible cause. There seem to be other people in this and the other thread saying that the fix didn’t work.