OK. I can live with that. I changed the OP to look in /usr/local/clang7/lib
rather than $(R_HOME)/lib
.
@bgoodri Are you on Stan’s Slack? I could pop in there.
I’m fine with adding a modification to ~/.R/Makevars
that contains:
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
Let me know your preference.
I haven’t joined Stan’s slack because I’m afraid I would never be able to leave, but if you want to talk off Discourse, I’m happy to do that.
Adding
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
seems to work for Stan, but I’m not sure it is the ultimate solution. First, people using Rcpp::sourceCpp
or inline::cxxfunction
on Catalina will still get errors from exceptions by default using C++11 or C++98. You could make the broader change
CXX = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXXLD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX11 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX11LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
CXX17 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX17LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
but I don’t know if that would adversely affect other packages that are implicitly relying on XCode to link. Stan, unlike almost all packages, enforces C++14 so only doing it for CXX14 and SHLIB_CXX14LD is unlikely to affect too many other packages.
More broadly, I don’t know why SHLIB_CXX{98,11,14,17}LD
on a Mac differs from the corresponding CXX{98,11,14,17}
. On Linux, they seem to be the same. But I stumbled into this workaround by observing on a friend’s Mac that (in a shell)
`R CMD config SHLIB_CXX14LD` --version
was clang++
from XCode, which was the reason for the conflict between LLVM-clang++ compiled code and XCode-libc++ . So, maybe R is just misconfigured for Macs?
I also don’t know why -L/usr/local/clang7/lib
works for us but -L$(R_HOME)/lib
does not work for us. The latter works for Rcpp::sourceCpp
but the former works for inline::cxxfunction
, which is what Stan calls. So, maybe the $(R_HOME)/lib/libc++.1.dylib
that comes with the R binary for Mac was compiled by XCode or is wrong for some other weird reason.
BTW, I don’t think you have to upgrade to Catalina yourself to test whether there are issues with any of the above. You just need to compile / link some C++ code and see what libc++.1.dylib
is picked up with @kevinushey’s test:
dlls <- getLoadedDLLs()
paths <- vapply(dlls, `[[`, "path", FUN.VALUE = character(1))
invisible(lapply(paths, function(path) {
if (!file.exists(path))
return(FALSE)
output <- system(paste("otool -L", shQuote(path), "| grep libc++ || true"),
intern = TRUE)
if (length(output) == 0)
return(FALSE)
writeLines(paste0(path, ":"))
writeLines(output)
}))
Probably should of chatted off-discourse at the start, but in the future, feel free to ping me via e-mail (@illinois.edu) if something arises.
I think so. We might want to ping Simon. Though, he’s a bit busy. But, I’d like to avoid a repeat of this when R 4.0.0 is released in April. The toolchain is slated to move to clang8
then.
I’ll investigate it a bit more.
I’m wondering if the reason for this is because $(R_HOME)
isn’t set without enclosing the statement with backticks, e.g. :
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L`$(R_HOME)/lib/libc++.1.dylib`
@torkar got $(R_HOME)
to expand to the correct path without the backticks but that was somehow insufficient to get inline::cxxfunction
to not link against /usr/lib/libc++.1.dylib
which continues to be a mystery to me.
Anyway, I agree it would be good to get someone from CRAN to weigh in. I would be happy to help further, but all I know is collected at
Hi!
I think I got a solution. Turns out that the ABI seems to have changed. So if I have this in ~/.R/Makevars
:
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX11LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
Then I can compile the exception example and get the desired output. The key line above is the SHLIB_CXXLDFLAGS
. The rpath instructs the linker to hard-code the path to the libraries found in the directory of R. Next I am hard-coding the path of the libc++abi.1.dylib
as used by R. Doing so seems to resolve the problems.
So the above should fix our problems. I just reinstalled Rcpp, inline and RcppParallel as binary packages from CRAN. Then I installed rstan and the mvn_orange example does now sample just fine.
Can others please verify?
I think this resolves the issue - I have in the meantime started a thread on the Mac-SIG mailinglist which does not include the latest info here. A ABI compiler change is a really nasty thing… in particular as Apple apparently did not change the library names. So this is probably a bug in Apple’s shipped clang from my perspective (rather than a feature).
Hi,
everything seems to work now using @coatless’s instruction and this .R/Makevars
:
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib
And, thanks to all the people who’ve spent the last month looking into this… :)
Hi!
Oh, I missed that this solves the issue as well.
Forcing the use of the R ABI libraries as suggested by myself even works with the XCode clang compiler as I just tried out with the exception example.
It’s probably cleaner to go with clang7 only, but at least people are not forced into the clang7 install.
Sebastian
I’ve verified that both the solution given by @bgoodri and @wds15’s addition of SHLIB_CXXLDFLAGS
works. I think @wds15’s approach is probably better as it: 1. relies less on specifying defaults for each language specification (c++{98, 11, 14,17}
) and 2. uses R’s shipped ABI.
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
SHLIB_CXXLDFLAGS+=-Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib
Before I roll out a new installer, is there any objection? @bgoodri?
I don’t know enough about Macs to object. I guess it should have some $(R_HOME)
stuff rather than /Library/Frameworks/R.framework/Resources/
I do wonder, however, what the implications for a src/Makevars file in a package like rstanarm or RBesT that comes with Stan programs that are compiled on CRAN. Should the src/Makevars file also have -Wl,-rpath,$(R_HOME)/lib $(R_HOME)/lib/libc++abi.1.dylib
somewhere in it? What if the user has installed R someplace else?
The following works:
# clang: start
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
# clang: end
Output from test:
R> Rcpp::sourceCpp(code =
+ ' #include <Rcpp.h>
+ using namespace Rcpp;
+ // [[Rcpp::export]]
+ int throw_exception() {
+ std::stringstream errmsg; errmsg << "this is an exception";
+ throw std::domain_error(errmsg.str());
+ return 0;
+ }
+ ', verbose = TRUE, rebuild = TRUE)
Generated extern "C" functions
--------------------------------------------------------
#include <Rcpp.h>
// throw_exception
int throw_exception();
RcppExport SEXP sourceCpp_12_throw_exception() {
BEGIN_RCPP
Rcpp::RObject rcpp_result_gen;
Rcpp::RNGScope rcpp_rngScope_gen;
rcpp_result_gen = Rcpp::wrap(throw_exception());
return rcpp_result_gen;
END_RCPP
}
Generated R functions
-------------------------------------------------------
`.sourceCpp_12_DLLInfo` <- dyn.load('/private/var/folders/b0/vt_1hj2d6yd8myx9lwh81pww0000gn/T/RtmpDGcrdF/sourceCpp-x86_64-apple-darwin15.6.0-1.0.2/sourcecpp_edec7e57c198/sourceCpp_14.so')
throw_exception <- Rcpp:::sourceCppFunction(function() {}, FALSE, `.sourceCpp_12_DLLInfo`, 'sourceCpp_12_throw_exception')
rm(`.sourceCpp_12_DLLInfo`)
Building shared library
--------------------------------------------------------
DIR: /private/var/folders/b0/vt_1hj2d6yd8myx9lwh81pww0000gn/T/RtmpDGcrdF/sourceCpp-x86_64-apple-darwin15.6.0-1.0.2/sourcecpp_edec7e57c198
/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB -o 'sourceCpp_14.so' --preclean 'fileedec67dd7d75.cpp'
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Rcpp/include" -I"/private/var/folders/b0/vt_1hj2d6yd8myx9lwh81pww0000gn/T/RtmpDGcrdF/sourceCpp-x86_64-apple-darwin15.6.0-1.0.2" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c fileedec67dd7d75.cpp -o fileedec67dd7d75.o
clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -Wl,-rpath,/Library/Frameworks/R.framework/Resources/lib /Library/Frameworks/R.framework/Resources/lib/libc++abi.1.dylib -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o sourceCpp_14.so fileedec67dd7d75.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
R>
R> throw_exception()
Error in throw_exception() : this is an exception
If there no objections, I’ll re-roll the installer based on this.
Released a new version of the installer here:
Special thanks to @bgoodri, @wds15, and @kevinushey for their efforts in helping identify problem and propose solutions.
Hi All,
First of all, thanks for all the hard work you put into this. I read through the other Catalina thread and this one, and although I think I followed all the steps, I somehow still get errors when running a Stan model through rstan (one that ran without errors in the previous version of Mac OS). These are the steps I took:
-
“upgrade” to Catalina (unfortunately). I’m now on Mac OS 10.15.1 (19B88)
-
remove rstan and StanHeaders (through clicking the ‘X’ in the RStudio packages pane)
-
in terminal:
xcode-select --install
-
install the @coatless r-macos-rtools from the post above. My Makevars file is now as follows:
# clang: start
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include
SHLIB_CXXLDFLAGS+=-Wl,-rpath,${R_HOME}/lib ${R_HOME}/lib/libc++abi.1.dylib
# clang: end
-
confirm that the following works:
library(readr) # or another package with compiled C++
Rcpp::sourceCpp(code =
'
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
int throw_exception() {
std::stringstream errmsg; errmsg << "this is an exception";
throw std::domain_error(errmsg.str());
return 0;
}
'
)
throw_exception()
(i.e. output is Error in throw_exception() : this is an exception
)
6. install.packages("rstan", type = "source", dependencies = TRUE)
7. run a model through rstan. Error messages:
[1] "Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)"
Somehow 2 out of the 4 chains run fine while the other 2 give this error.
Can someone please help me out? I hope that I simply missed a crucial step in one of the threads. If not, please let me know what I can do to provide more information.
Thanks!
Maurits
If 2 of the 4 chains, never throw exceptions, that could happen. What happens when you run that model after putting
CXX14 = /usr/local/clang7/bin/clang++ -stdlib=libc++ -nostdinc++ -I/usr/local/clang7/include/c++/v1
SHLIB_CXX14LD = /usr/local/clang7/bin/clang++ -L/usr/local/clang7/lib/
into ~/.R/Makevars ?
Thanks for the quick reply. After updating Makevars (I added these lines without touching the rest) I get the exact same behavior - 2 chains run fine, 2 chains crash immediately with the error message from before.
Missing the ~/.Renviron
file…
# clang: start
PATH="/usr/local/clang7/bin:${PATH}"
# clang: end
I just checked - I do have a ~/.Renviron file with the exact same content.
Did you restart your R session? Make sure that you are not loading an old R environment.
Please also verify the installer package locations as this was done outside of the installer.
ls /usr/local
Thanks for all the quick replies!
ls /usr/local
gives
Cellar bin clang6 clang8 f-secure include man remotedesktop texlive
Homebrew clang4 clang7 etc gfortran lib opt share var
See anything wrong?
Interestingly, I restarted R and removed the saved Stan model (.rds file) again. This time when running it tells me Building R package from source requires installation of additional build tools. Do you want ot install the additional tools now?
Since I had removed the .rds files before posting here, this change must have come from updating the Makevars file with @bgoodri 's post. Any ideas what I’m missing now?
xcode-select --install
tells me the command line tools are already installed.
Something is off on your system. I’m not sure what though.
Could you give me:
- A screenshot of the startup text / RStudio IDE panel
- Details about your R installation via:
utils::sessionInfo()