Dealing with Catalina II

Now it sampled correctly…
Your DLL code says this:

/Users/torkarr/Library/R/3.6/library/Rcpp/libs/Rcpp.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.6.0)
/Users/torkarr/Library/R/3.6/library/scales/libs/scales.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/tidyselect/libs/tidyselect.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/dplyr/libs/dplyr.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/rstan/libs/rstan.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//RtmpDpq1C9/file9780549a963b.so:
/usr/local/clang7/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)

And compilation arguments are:

Compilation argument:
/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB file9780549a963b.cpp 2> file9780549a963b.cpp.err.txt
/usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Users/torkarr/Library/R/3.6/library/Rcpp/include/" -I"/Users/torkarr/Library/R/3.6/library/RcppEigen/include/" -I"/Users/torkarr/Library/R/3.6/library/RcppEigen/include/unsupported" -I"/Users/torkarr/Library/R/3.6/library/BH/include" -I"/Users/torkarr/Library/R/3.6/library/StanHeaders/include/src/" -I"/Users/torkarr/Library/R/3.6/library/StanHeaders/include/" -I"/Users/torkarr/Library/R/3.6/library/rstan/include" -DEIGEN_NO_DEBUG -DBOOST_DISABLE_ASSERTS -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -Wall -g -O2 -c file9780549a963b.cpp -o file9780549a963b.o
/usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o file9780549a963b.so file9780549a963b.o -L/Users/torkarr/Library/R/3.6/library/StanHeaders/lib/ -lStanHeaders -F/Library/Frameworks/R.framework/… -framework R -Wl,-framework -Wl,CoreFoundation

I need to fetch my kids now so will check later this evening (Swedish time).

Is /Library/Frameworks/R.framework/Resources/lib/libc++1.dylib a symlink to /usr/lib/libc++.1.dylib in Sweden?

Not sure if it helps at all, but in case any linux-based devs want to get hands-on with Catalina, I just found these instructions for setting it up as a VM (inc. auto-downloading the OS ISO): https://github.com/foxlet/macOS-Simple-KVM/blob/master/README.md

1 Like

Well, in Sweden there’s no libc++1.dylib ;) But I have a libc++.1.dylib and it’s not a link:

% ls -lsa libc++.1.dylib
1656 -rwxrwxr-x 1 root admin 845100 Jul 6 04:12 libc++.1.dylib

OK. Try agin with

-L$(R_HOME)/lib

in ~/.R/Makevars instead of

-L/usr/local/clang7/lib

Got an unknown reason during sampling again. Previously it sampled nicely with -L/usr/local/clang7/lib

Output below:

Compilation argument:
/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB fileac4a36d3023f.cpp 2> fileac4a36d3023f.cpp.err.txt
/usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Users/torkarr/Library/R/3.6/library/Rcpp/include/" -I"/Users/torkarr/Library/R/3.6/library/RcppEigen/include/" -I"/Users/torkarr/Library/R/3.6/library/RcppEigen/include/unsupported" -I"/Users/torkarr/Library/R/3.6/library/BH/include" -I"/Users/torkarr/Library/R/3.6/library/StanHeaders/include/src/" -I"/Users/torkarr/Library/R/3.6/library/StanHeaders/include/" -I"/Users/torkarr/Library/R/3.6/library/rstan/include" -DEIGEN_NO_DEBUG -DBOOST_DISABLE_ASSERTS -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -Wall -g -O2 -c fileac4a36d3023f.cpp -o fileac4a36d3023f.o
/usr/local/clang7/bin/clang++ -L/Library/Frameworks/R.framework/Resources/lib -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o fileac4a36d3023f.so fileac4a36d3023f.o -L/Users/torkarr/Library/R/3.6/library/StanHeaders/lib/ -lStanHeaders -F/Library/Frameworks/R.framework/… -framework R -Wl,-framework -Wl,CoreFoundation

and,

/Users/torkarr/Library/R/3.6/library/Rcpp/libs/Rcpp.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.6.0)
/Users/torkarr/Library/R/3.6/library/scales/libs/scales.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/tidyselect/libs/tidyselect.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/dplyr/libs/dplyr.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/rstan/libs/rstan.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//Rtmp6fAMsg/fileac4a36d3023f.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)

Can you add a -lc++ at the end of

SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L$(R_HOME)/lib/"

in ~/.R/Makevars .

Same error with the sampling.

Compilation argument:
/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB fileb7c99ed272e.cpp 2> fileb7c99ed272e.cpp.err.txt
/usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Users/torkarr/Library/R/3.6/library/Rcpp/include/" -I"/Users/torkarr/Library/R/3.6/library/RcppEigen/include/" -I"/Users/torkarr/Library/R/3.6/library/RcppEigen/include/unsupported" -I"/Users/torkarr/Library/R/3.6/library/BH/include" -I"/Users/torkarr/Library/R/3.6/library/StanHeaders/include/src/" -I"/Users/torkarr/Library/R/3.6/library/StanHeaders/include/" -I"/Users/torkarr/Library/R/3.6/library/rstan/include" -DEIGEN_NO_DEBUG -DBOOST_DISABLE_ASSERTS -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -Wall -g -O2 -c fileb7c99ed272e.cpp -o fileb7c99ed272e.o
/usr/local/clang7/bin/clang++ -L/Library/Frameworks/R.framework/Resources/lib -lc++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o fileb7c99ed272e.so fileb7c99ed272e.o -L/Users/torkarr/Library/R/3.6/library/StanHeaders/lib/ -lStanHeaders -F/Library/Frameworks/R.framework/… -framework R -Wl,-framework -Wl,CoreFoundation

/Users/torkarr/Library/R/3.6/library/Rcpp/libs/Rcpp.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.6.0)
/Users/torkarr/Library/R/3.6/library/scales/libs/scales.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/tidyselect/libs/tidyselect.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/dplyr/libs/dplyr.so:
/Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/Users/torkarr/Library/R/3.6/library/rstan/libs/rstan.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//RtmpaaeUD3/fileb7c99ed272e.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)

And

example(cxxfunction, package = "inline", run.dontrun = TRUE)

links the DLLs to /usr/lib/libc++.1.dylib or to /Library/Frameworks/R.framework/Resources/lib/libc++.1.dylib?

I ran example() and then your DLL function

/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//RtmpPbjB1G/filec6655bfafc7a.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)
/Users/torkarr/Library/R/3.6/library/Rcpp/libs/Rcpp.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.6.0)
/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//RtmpPbjB1G/filec6653ad5dcc3.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)
/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//RtmpPbjB1G/filec6652e90a462.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)
/Users/torkarr/Library/R/3.6/library/RcppArmadillo/libs/RcppArmadillo.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.6.0)
/var/folders/5p/dlcj_kd94bx83r83k_d77j54pspdk5/T//RtmpPbjB1G/filec6651a1835d0.so:
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)

OK. I can live with that. I changed the OP to look in /usr/local/clang7/lib rather than $(R_HOME)/lib.

1 Like

@bgoodri Are you on Stan’s Slack? I could pop in there.

I’m fine with adding a modification to ~/.R/Makevars that contains:

CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

Let me know your preference.

I haven’t joined Stan’s slack because I’m afraid I would never be able to leave, but if you want to talk off Discourse, I’m happy to do that.

Adding

CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

seems to work for Stan, but I’m not sure it is the ultimate solution. First, people using Rcpp::sourceCpp or inline::cxxfunction on Catalina will still get errors from exceptions by default using C++11 or C++98. You could make the broader change

CXX = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXXLD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX11 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX11LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX17 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX17LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/

but I don’t know if that would adversely affect other packages that are implicitly relying on XCode to link. Stan, unlike almost all packages, enforces C++14 so only doing it for CXX14 and SHLIB_CXX14LD is unlikely to affect too many other packages.

More broadly, I don’t know why SHLIB_CXX{98,11,14,17}LD on a Mac differs from the corresponding CXX{98,11,14,17}. On Linux, they seem to be the same. But I stumbled into this workaround by observing on a friend’s Mac that (in a shell)

`R CMD config SHLIB_CXX14LD` --version

was clang++ from XCode, which was the reason for the conflict between LLVM-clang++ compiled code and XCode-libc++ . So, maybe R is just misconfigured for Macs?

I also don’t know why -L/usr/local/clang7/lib works for us but -L$(R_HOME)/lib does not work for us. The latter works for Rcpp::sourceCpp but the former works for inline::cxxfunction, which is what Stan calls. So, maybe the $(R_HOME)/lib/libc++.1.dylib that comes with the R binary for Mac was compiled by XCode or is wrong for some other weird reason.

BTW, I don’t think you have to upgrade to Catalina yourself to test whether there are issues with any of the above. You just need to compile / link some C++ code and see what libc++.1.dylib is picked up with @kevinushey’s test:

dlls <- getLoadedDLLs()
paths <- vapply(dlls, `[[`, "path", FUN.VALUE = character(1))
invisible(lapply(paths, function(path) {
  
  if (!file.exists(path))
    return(FALSE)
  
  output <- system(paste("otool -L", shQuote(path), "| grep libc++ || true"),
                   intern = TRUE)
  if (length(output) == 0)
    return(FALSE)
  
  writeLines(paste0(path, ":"))
  writeLines(output)
  
}))

Probably should of chatted off-discourse at the start, but in the future, feel free to ping me via e-mail (@illinois.edu) if something arises.

I think so. We might want to ping Simon. Though, he’s a bit busy. But, I’d like to avoid a repeat of this when R 4.0.0 is released in April. The toolchain is slated to move to clang8 then.

I’ll investigate it a bit more.

I’m wondering if the reason for this is because $(R_HOME) isn’t set without enclosing the statement with backticks, e.g. :

SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L`$(R_HOME)/lib/libc++.1.dylib`
1 Like

@torkar got $(R_HOME) to expand to the correct path without the backticks but that was somehow insufficient to get inline::cxxfunction to not link against /usr/lib/libc++.1.dylib which continues to be a mystery to me.

Anyway, I agree it would be good to get someone from CRAN to weigh in. I would be happy to help further, but all I know is collected at

Hi!

I think I got a solution. Turns out that the ABI seems to have changed. So if I have this in ~/.R/Makevars:

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX11LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib

Then I can compile the exception example and get the desired output. The key line above is the SHLIB_CXXLDFLAGS. The rpath instructs the linker to hard-code the path to the libraries found in the directory of R. Next I am hard-coding the path of the libc++abi.1.dylib as used by R. Doing so seems to resolve the problems.

So the above should fix our problems. I just reinstalled Rcpp, inline and RcppParallel as binary packages from CRAN. Then I installed rstan and the mvn_orange example does now sample just fine.

Can others please verify?

I think this resolves the issue - I have in the meantime started a thread on the Mac-SIG mailinglist which does not include the latest info here. A ABI compiler change is a really nasty thing… in particular as Apple apparently did not change the library names. So this is probably a bug in Apple’s shipped clang from my perspective (rather than a feature).

1 Like

Hi,

everything seems to work now using @coatless’s instruction and this .R/Makevars:

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib

And, thanks to all the people who’ve spent the last month looking into this… :)

Hi!

Oh, I missed that this solves the issue as well.

Forcing the use of the R ABI libraries as suggested by myself even works with the XCode clang compiler as I just tried out with the exception example.

It’s probably cleaner to go with clang7 only, but at least people are not forced into the clang7 install.

Sebastian

4 Likes

I’ve verified that both the solution given by @bgoodri and @wds15’s addition of SHLIB_CXXLDFLAGS works. I think @wds15’s approach is probably better as it: 1. relies less on specifying defaults for each language specification (c++{98, 11, 14,17}) and 2. uses R’s shipped ABI.

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib

Before I roll out a new installer, is there any objection? @bgoodri?

1 Like

I don’t know enough about Macs to object. I guess it should have some $(R_HOME) stuff rather than /Library/Frameworks/R.framework/Resources/

I do wonder, however, what the implications for a src/Makevars file in a package like rstanarm or RBesT that comes with Stan programs that are compiled on CRAN. Should the src/Makevars file also have -Wl,-rpath,$(R_HOME)/lib $(R_HOME)/lib/libc++abi.1.dylib somewhere in it? What if the user has installed R someplace else?