Dealing with Catalina II

OK. I can live with that. I changed the OP to look in /usr/local/clang7/lib rather than $(R_HOME)/lib.

1 Like

@bgoodri Are you on Stan’s Slack? I could pop in there.

I’m fine with adding a modification to ~/.R/Makevars that contains:

CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

Let me know your preference.

I haven’t joined Stan’s slack because I’m afraid I would never be able to leave, but if you want to talk off Discourse, I’m happy to do that.

Adding

CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

seems to work for Stan, but I’m not sure it is the ultimate solution. First, people using Rcpp::sourceCpp or inline::cxxfunction on Catalina will still get errors from exceptions by default using C++11 or C++98. You could make the broader change

CXX = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXXLD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX11 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX11LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX17 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX17LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

but I don’t know if that would adversely affect other packages that are implicitly relying on XCode to link. Stan, unlike almost all packages, enforces C++14 so only doing it for CXX14 and SHLIB_CXX14LD is unlikely to affect too many other packages.

More broadly, I don’t know why SHLIB_CXX{98,11,14,17}LD on a Mac differs from the corresponding CXX{98,11,14,17}. On Linux, they seem to be the same. But I stumbled into this workaround by observing on a friend’s Mac that (in a shell)

`R CMD config SHLIB_CXX14LD` --version

was clang++ from XCode, which was the reason for the conflict between LLVM-clang++ compiled code and XCode-libc++ . So, maybe R is just misconfigured for Macs?

I also don’t know why -L/usr/local/clang7/lib works for us but -L$(R_HOME)/lib does not work for us. The latter works for Rcpp::sourceCpp but the former works for inline::cxxfunction, which is what Stan calls. So, maybe the $(R_HOME)/lib/libc++.1.dylib that comes with the R binary for Mac was compiled by XCode or is wrong for some other weird reason.

BTW, I don’t think you have to upgrade to Catalina yourself to test whether there are issues with any of the above. You just need to compile / link some C++ code and see what libc++.1.dylib is picked up with @kevinushey’s test:

dlls <- getLoadedDLLs()
paths <- vapply(dlls, `[[`, "path", FUN.VALUE = character(1))
invisible(lapply(paths, function(path) {
  
  if (!file.exists(path))
    return(FALSE)
  
  output <- system(paste("otool -L", shQuote(path), "| grep libc++ || true"),
                   intern = TRUE)
  if (length(output) == 0)
    return(FALSE)
  
  writeLines(paste0(path, ":"))
  writeLines(output)
  
}))

Probably should of chatted off-discourse at the start, but in the future, feel free to ping me via e-mail (@illinois.edu) if something arises.

I think so. We might want to ping Simon. Though, he’s a bit busy. But, I’d like to avoid a repeat of this when R 4.0.0 is released in April. The toolchain is slated to move to clang8 then.

I’ll investigate it a bit more.

I’m wondering if the reason for this is because $(R_HOME) isn’t set without enclosing the statement with backticks, e.g. :

SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L`$(R_HOME)/lib/libc++.1.dylib`
1 Like

@torkar got $(R_HOME) to expand to the correct path without the backticks but that was somehow insufficient to get inline::cxxfunction to not link against /usr/lib/libc++.1.dylib which continues to be a mystery to me.

Anyway, I agree it would be good to get someone from CRAN to weigh in. I would be happy to help further, but all I know is collected at

Hi!

I think I got a solution. Turns out that the ABI seems to have changed. So if I have this in ~/.R/Makevars:

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX11LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib

Then I can compile the exception example and get the desired output. The key line above is the SHLIB_CXXLDFLAGS. The rpath instructs the linker to hard-code the path to the libraries found in the directory of R. Next I am hard-coding the path of the libc++abi.1.dylib as used by R. Doing so seems to resolve the problems.

So the above should fix our problems. I just reinstalled Rcpp, inline and RcppParallel as binary packages from CRAN. Then I installed rstan and the mvn_orange example does now sample just fine.

Can others please verify?

I think this resolves the issue - I have in the meantime started a thread on the Mac-SIG mailinglist which does not include the latest info here. A ABI compiler change is a really nasty thing… in particular as Apple apparently did not change the library names. So this is probably a bug in Apple’s shipped clang from my perspective (rather than a feature).

1 Like

Hi,

everything seems to work now using @coatless’s instruction and this .R/Makevars:

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib

And, thanks to all the people who’ve spent the last month looking into this… :)

Hi!

Oh, I missed that this solves the issue as well.

Forcing the use of the R ABI libraries as suggested by myself even works with the XCode clang compiler as I just tried out with the exception example.

It’s probably cleaner to go with clang7 only, but at least people are not forced into the clang7 install.

Sebastian

4 Likes

I’ve verified that both the solution given by @bgoodri and @wds15’s addition of SHLIB_CXXLDFLAGS works. I think @wds15’s approach is probably better as it: 1. relies less on specifying defaults for each language specification (c++{98, 11, 14,17}) and 2. uses R’s shipped ABI.

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib

Before I roll out a new installer, is there any objection? @bgoodri?

1 Like

I don’t know enough about Macs to object. I guess it should have some $(R_HOME) stuff rather than /Library/Frameworks/R.framework/Resources/

I do wonder, however, what the implications for a src/Makevars file in a package like rstanarm or RBesT that comes with Stan programs that are compiled on CRAN. Should the src/Makevars file also have -Wl,-rpath,$(R_HOME)/lib $(R_HOME)/lib/libc++abi.1.dylib somewhere in it? What if the user has installed R someplace else?

The following works:

# clang: start
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
# clang: end

Output from test:

R> Rcpp::sourceCpp(code = 
+ ' #include <Rcpp.h>
+ using namespace Rcpp;  
+ // [[Rcpp::export]]
+ int throw_exception() { 
+   std::stringstream errmsg; errmsg << "this is an exception";
+   throw std::domain_error(errmsg.str()); 
+   return 0;
+ }
+ ', verbose = TRUE, rebuild = TRUE)

Generated extern "C" functions 
--------------------------------------------------------


#include <Rcpp.h>
// throw_exception
int throw_exception();
RcppExport SEXP sourceCpp_12_throw_exception() {
BEGIN_RCPP
    Rcpp::RObject rcpp_result_gen;
    Rcpp::RNGScope rcpp_rngScope_gen;
    rcpp_result_gen = Rcpp::wrap(throw_exception());
    return rcpp_result_gen;
END_RCPP
}

Generated R functions 
-------------------------------------------------------

`.sourceCpp_12_DLLInfo` <- dyn.load('/private/var/folders/b0/vt_1hj2d6yd8myx9lwh81pww0000gn/T/RtmpDGcrdF/sourceCpp-x86_64-apple-darwin15.6.0-1.0.2/sourcecpp_edec7e57c198/sourceCpp_14.so')

throw_exception <- Rcpp:::sourceCppFunction(function() {}, FALSE, `.sourceCpp_12_DLLInfo`, 'sourceCpp_12_throw_exception')

rm(`.sourceCpp_12_DLLInfo`)

Building shared library
--------------------------------------------------------

DIR: /private/var/folders/b0/vt_1hj2d6yd8myx9lwh81pww0000gn/T/RtmpDGcrdF/sourceCpp-x86_64-apple-darwin15.6.0-1.0.2/sourcecpp_edec7e57c198

/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB -o 'sourceCpp_14.so' --preclean  'fileedec67dd7d75.cpp'  
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Rcpp/include" -I"/private/var/folders/b0/vt_1hj2d6yd8myx9lwh81pww0000gn/T/RtmpDGcrdF/sourceCpp-x86_64-apple-darwin15.6.0-1.0.2" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include  -fPIC  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c fileedec67dd7d75.cpp -o fileedec67dd7d75.o
clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o sourceCpp_14.so fileedec67dd7d75.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
R> 
R> throw_exception()
Error in throw_exception() : this is an exception

If there no objections, I’ll re-roll the installer based on this.

Released a new version of the installer here:

Special thanks to @bgoodri, @wds15, and @kevinushey for their efforts in helping identify problem and propose solutions.

6 Likes

Hi All,

First of all, thanks for all the hard work you put into this. I read through the other Catalina thread and this one, and although I think I followed all the steps, I somehow still get errors when running a Stan model through rstan (one that ran without errors in the previous version of Mac OS). These are the steps I took:

  1. “upgrade” to Catalina (unfortunately). I’m now on Mac OS 10.15.1 (19B88)

  2. remove rstan and StanHeaders (through clicking the ‘X’ in the RStudio packages pane)

  3. in terminal: xcode-select --install

  4. install the @coatless r-macos-rtools from the post above. My Makevars file is now as follows:

    # clang: start
    CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
    CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
    CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
    CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

    SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
    # clang: end

  5. confirm that the following works:

library(readr) # or another package with compiled C++

Rcpp::sourceCpp(code = 
'
#include <Rcpp.h>
using namespace Rcpp; 

// [[Rcpp::export]]
int throw_exception() { 
  std::stringstream errmsg; errmsg << "this is an exception";
  throw std::domain_error(errmsg.str()); 
  return 0;
}
'
)

throw_exception()

(i.e. output is Error in throw_exception() : this is an exception )
6. install.packages("rstan", type = "source", dependencies = TRUE)
7. run a model through rstan. Error messages:

[1] "Error in sampler$call_sampler(args_list[[i]]) : "
[2] "  c++ exception (unknown reason)"     

Somehow 2 out of the 4 chains run fine while the other 2 give this error.

Can someone please help me out? I hope that I simply missed a crucial step in one of the threads. If not, please let me know what I can do to provide more information.

Thanks!
Maurits

If 2 of the 4 chains, never throw exceptions, that could happen. What happens when you run that model after putting

CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

into ~/.R/Makevars ?

1 Like

Thanks for the quick reply. After updating Makevars (I added these lines without touching the rest) I get the exact same behavior - 2 chains run fine, 2 chains crash immediately with the error message from before.

Missing the ~/.Renviron file…

# clang: start
PATH="/usr/local/clang7/bin:${PATH}"
# clang: end
1 Like

I just checked - I do have a ~/.Renviron file with the exact same content.

Did you restart your R session? Make sure that you are not loading an old R environment.

Please also verify the installer package locations as this was done outside of the installer.

ls /usr/local
1 Like

Thanks for all the quick replies!

ls /usr/local gives

Cellar bin clang6 clang8 f-secure include man remotedesktop texlive
Homebrew clang4 clang7 etc gfortran lib opt share var

See anything wrong?

Interestingly, I restarted R and removed the saved Stan model (.rds file) again. This time when running it tells me Building R package from source requires installation of additional build tools. Do you want ot install the additional tools now?

Since I had removed the .rds files before posting here, this change must have come from updating the Makevars file with @bgoodri 's post. Any ideas what I’m missing now?
xcode-select --install tells me the command line tools are already installed.

Something is off on your system. I’m not sure what though.

Could you give me:

  1. A screenshot of the startup text / RStudio IDE panel
  2. Details about your R installation via:
utils::sessionInfo()