The warnings being displayed by the configure script have been around for a while. In short, they are nothing to worry about. The g++ warning is because we’re using clang instead of gcc and the LAPACK version is because we’re by default using a crippled LAPACK that ships with R – unless one compiles R from scratch with a link out to MKL et cetera.
For in-depth remarks, please see:
How does it “not” work? I’m missing this part. Very difficult for me to troubleshoot if I do not know what isn’t working.
I’m having the same issues after installing Catalina, and after following @coatless instructions for setting up the tool chain. The issue,
[1] "Error in sampler$call_sampler(args_list[[i]]) : " " c++ exception (unknown reason)"
occurs after an apparent successful compile of the code and when in the sampling step.
If it helps, re-running the example post <- stan(...), results in a slightly different error,
Chain 1: empty_nested() must be true before calling recover_memory()
[1] "Error in sampler$call_sampler(args_list[[i]]) : "
[2] " empty_nested() must be true before calling recover_memory()"
The error from the example also occurs in my own Stan programs, so probably not unique to the example. Here’s the example in full from the posts above:
library(rstan); library(brms)
output <- c(0.5237868884927709,0.9487179487179488,0.5494563370710159,0.46386348238482383,0.4458125179443009,0.9830952380952381,0.9685185185185186,0.5118241235888294,0.8824457593688363,0.5644325518178729,0.6317411924119242,0.4893159632500718,0.9416071428571429,0.9145833333333333,0.4509011685482273,0.7682445759368836,0.2748216106014271,0.7207063008130081,0.6109249005373036,0.9156547619047619,0.9011574074074074,0.6936423054070112,0.9012820512820513,0.8805895344886171,0.7585789295392954,0.6982066978384808,0.9988095238095238,0.9606481481481481,0.7225985343632402,0.9347140039447732,0.7936459395174992,0.7401422764227643,0.7547029449161232,0.9897619047619047,0.9571759259259259,0.734383046147752,1.0,0.8722222222222221,0.6707232384823848,0.6119692588491038,1.0,1.0,0.5487621311150723,0.8960552268244577,0.6401885830784912,0.7439871273712737,0.6910643533899349,0.9875,0.9847222222222223,0.5955040602099425,0.9396449704142013,0.5083588175331294,0.6947069783197831,0.5883210286698658,0.9872023809523809,0.9643518518518519)
categs<-as.factor(rep(c("FOO1","FOO2","FOO3","FOO4","FOO5","FOO6","FOO7","FOO8"),each=7));
blocks <- c(1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7)
df1 <- data.frame(y=output, algorithm=as.integer(categs), block_id=blocks)
post <- stan(model_code = make_stancode(y ~ 1 + (1 | algorithm), data = df1),
data = make_standata(y ~ 1 + (1 | algorithm), data = df1),
verbose = TRUE)
In the r environment, I’m using clang7:
> system("clang++ --version")
clang version 7.0.0 (tags/RELEASE_700/final)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
InstalledDir: /usr/local/clang7/bin
My .R/Makevars:
# clang: start
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
# clang: end
CXX14FLAGS=-O3 -march=native -mtune=native
CXX14FLAGS += -arch x86_64 -ftemplate-depth-256
Though I have also tried removing the CXX14 lines above.
My R version is 3.6.1, and rstan (and brms):
> library(rstan); library(brms)
Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.19.2, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)
Loading required package: Rcpp
Registered S3 method overwritten by 'xts':
method from
as.zoo.xts zoo
Loading 'brms' package (version 2.10.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').
Attaching package: ‘brms’
The following object is masked from ‘package:rstan’:
loo
Here is the verbose dump.txt (61.8 KB) when executing stan() in the example above.
I haven’t taken the risk to update to Catalina because I’ve needed to be running models on my laptop pretty consistently recently. Mac never used to be a problem for us but now that there have been issues with both Mojave and Catalina we should probably invest in a Mac computer that we can keep updating as soon as there’s a Mac update and just use that machine to test installation and compilation without worrying about messing up personal computers.
@coatless this is affecting RStan and downstream packages because Stan branches on the type of exception being thrown but isn’t specific to RStan. We first encountered it with clang4 back in 2017 where doing
Rcpp::sourceCpp(code =
'
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
int throw_exception() {
std::stringstream errmsg; errmsg << "this is an exception";
throw std::domain_error(errmsg.str());
return 0;
}
'
)
throw_exception()
would yield
Error in throw_exception() : c++ exception (unknown reason)
rather than
Error in throw_exception() : this is an exception
We worked around this problem with clang4 from AT&T in RStan with a variation of a hack from Kevin Ushey that became popular:
to relink against XCode’s C++ library. But CRAN was never too happy about that.
I’m pretty sure we confirmed that the problem went away with clang6, but now it seems to be back with clang7 and / or Catalina. Do you have any ideas besides applying our previous hack again?
From running:
I’m getting:
CHECKING DATA AND PREPROCESSING FOR MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW.
COMPILING MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW.
STARTING SAMPLER FOR MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW.
SAMPLING FOR MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 2.5e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.25 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
Chain 1: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 1: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 1: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 1: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 1: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 1: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 1: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 1: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 1: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 1: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 1: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 1: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 1:
Chain 1: Elapsed Time: 0.227319 seconds (Warm-up)
Chain 1: 0.108264 seconds (Sampling)
Chain 1: 0.335583 seconds (Total)
Chain 1:
SAMPLING FOR MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW (CHAIN 2).
Chain 2:
Chain 2: Gradient evaluation took 1.4e-05 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.14 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2:
Chain 2:
Chain 2: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 2: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 2: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 2: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 2: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 2: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 2: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 2: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 2: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 2: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 2: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 2: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 2:
Chain 2: Elapsed Time: 0.207346 seconds (Warm-up)
Chain 2: 0.114258 seconds (Sampling)
Chain 2: 0.321604 seconds (Total)
Chain 2:
SAMPLING FOR MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW (CHAIN 3).
Chain 3:
Chain 3: Gradient evaluation took 1.2e-05 seconds
Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.12 seconds.
Chain 3: Adjust your expectations accordingly!
Chain 3:
Chain 3:
Chain 3: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 3: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 3: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 3: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 3: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 3: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 3: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 3: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 3: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 3: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 3: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 3: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 3:
Chain 3: Elapsed Time: 0.193264 seconds (Warm-up)
Chain 3: 0.112713 seconds (Sampling)
Chain 3: 0.305977 seconds (Total)
Chain 3:
SAMPLING FOR MODEL '4bd718ce37045da5116c1e4a2933dbfb' NOW (CHAIN 4).
Chain 4:
Chain 4: Gradient evaluation took 1.3e-05 seconds
Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0.13 seconds.
Chain 4: Adjust your expectations accordingly!
Chain 4:
Chain 4:
Chain 4: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 4: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 4: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 4: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 4: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 4: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 4: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 4: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 4: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 4: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 4: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 4: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 4:
Chain 4: Elapsed Time: 0.184546 seconds (Warm-up)
Chain 4: 0.110832 seconds (Sampling)
Chain 4: 0.295378 seconds (Total)
Chain 4:
Warning: There were 5 divergent transitions after warmup. Increasing adapt_delta above 0.8 may help. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
Warning: Examine the pairs() plot to diagnose sampling problems
Maybe the data is bad?
Maybe change to:
Yeah, that’s my only solution at the moment.
Can anyone with clang7 or clang8 but not Catalina run this
Rcpp::sourceCpp(code =
'
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
int throw_exception() {
std::stringstream errmsg; errmsg << "this is an exception";
throw std::domain_error(errmsg.str());
return 0;
}
'
)
throw_exception()
to see if you get the “(unknown reason)” thing?
I think the general plumbing recommended isn’t ideal anymore. Sample:
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
double takeLog(double val) {
try {
if (val <= 0.0) { // log() not defined here
throw std::range_error("Inadmissible value");
}
return log(val);
} catch(std::exception &ex) {
forward_exception_to_r(ex);
} catch(...) {
::Rf_error("c++ exception (unknown reason)");
}
return NA_REAL; // not reached
}
In particular, the forward_exception_to_r() is marked as an internal function. The exception article on Rcpp Gallery subsequently needs to be re-written.
c.f.
That makes some sense, but in our case the try ... catch is from the Stan code and we just create a module for it with RStan. So, it would be a challenge to get Rcpp-specific code in there. Nor does it explain why the hack of calling install_name_tool worked (with clang4 at least). Or why it may not be a problem with clang7 unless you also have Catalina.
Guys is this of any relevance here: https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features
Unfortunately, no. The issue is Apple stopped shipping an installer package for headers that R uses, which compiles with the fact that R is still looking for the headers in the old location.
Just an FYI: Using Catalina and everything installed according to @coatless’s instructions, if I run @bgoodri’s example or @coatless’s examples it doesn’t matter. I get the same error msg:
Error in throw_exception() : c++ exception (unknown reason)
Again, posting for reference.
I can run simple models it seems, like the examples here. But whenever I try to run rstanrm or a brms I get:
fit <- stan_glm(y ~ x1, data=dat)
SAMPLING FOR MODEL ‘continuous’ NOW (CHAIN 1).
Chain 1: empty_nested() must be true before calling recover_memory()
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " empty_nested() must be true before calling recover_memory()”
error occurred during calling the sampler; sampling not done
Error in check_stanfit(stanfit) :
Invalid stanfit object produced please report bug
or
brm(y ~ x1, data=dat)
Compiling the C++ model
recompiling to avoid crashing R session
Start sampling
SAMPLING FOR MODEL ‘b3e140f1aba3e9c1258901076a1ed29b’ NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 2.5e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.25 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)”
error occurred during calling the sampler; sampling not done
Family: gaussian
Links: mu = identity; sigma = identity
Formula: y ~ x1
Data: dat (Number of observations: 30)
The model does not contain posterior samples.
Thanks!
It looks like this is related to the Qs around this email chain. The next few messages talk about diagnosing is the problem is different libc++ versions
https://stat.ethz.ch/pipermail/r-sig-mac/2017-September/012491.html
Just a question: Do you want me to file this as an Issue on GitHub?
No
If you have Catalina and rstanarm yields any errors when you try to estimate a model. Try installing / testing it from source with
remotes::install_github("stan-dev/rstanarm", build_vignettes = FALSE)
library(rstanarm)
example(example_model)
plus whatever model was not working before.
If you have Catalina and Stan models are not working, try running this function on the object produced by stan_model:
fix_Mac <- function(sm) {
stopifnot(is(sm, "stanmodel"))
dso_last_path <- sm@dso@.CXXDSOMISC$dso_last_path
CLANG_DIR <- tail(n = 1, grep("clang[456789]",
x = list.dirs("/usr/local", recursive = FALSE),
value = TRUE))
if (length(CLANG_DIR) == 0L) stop("no clang from CRAN found")
LIBCPP <- file.path(CLANG_DIR, "lib", "libc++.1.dylib")
if (!file.exists(LIBCPP)) stop("no unique libc++.1?.dylib found")
Rv <- R.version
GOOD <- file.path("/Library", "Frameworks", "R.framework", "Versions",
paste(Rv$major, substr(Rv$minor, 1, 1), sep = "."),
"Resources", "lib", "libc++.1.dylib")
if (!file.exists(GOOD)) stop(paste(GOOD, "not found"))
cmd <- paste(
"install_name_tool",
"-change",
LIBCPP,
GOOD,
dso_last_path
)
system(cmd)
dyn.unload(dso_last_path)
dyn.load(dso_last_path)
return(invisible(NULL))
}
followed by sampling(...).