Dealing with Catalina

coatless · October 10, 2019, 6:42pm

Maybe change to:

bgoodri · October 10, 2019, 6:44pm

Yeah, that’s my only solution at the moment.

bgoodri · October 10, 2019, 6:47pm

Can anyone with clang7 or clang8 but not Catalina run this

Rcpp::sourceCpp(code = 
'
#include <Rcpp.h>
using namespace Rcpp; 

// [[Rcpp::export]]
int throw_exception() { 
  std::stringstream errmsg; errmsg << "this is an exception";
  throw std::domain_error(errmsg.str()); 
  return 0;
}
'
)

throw_exception()

to see if you get the “(unknown reason)” thing?

coatless · October 10, 2019, 6:55pm

I think the general plumbing recommended isn’t ideal anymore. Sample:

#include <Rcpp.h>

using namespace Rcpp;
 
// [[Rcpp::export]]
double takeLog(double val) {
    try {
        if (val <= 0.0) {         	// log() not defined here
            throw std::range_error("Inadmissible value");
        }
        return log(val);
    } catch(std::exception &ex) {	
	forward_exception_to_r(ex);
    } catch(...) { 
	::Rf_error("c++ exception (unknown reason)"); 
    }
    return NA_REAL;             // not reached
}

In particular, the forward_exception_to_r() is marked as an internal function. The exception article on Rcpp Gallery subsequently needs to be re-written.

c.f.

bgoodri · October 10, 2019, 7:17pm

That makes some sense, but in our case the try ... catch is from the Stan code and we just create a module for it with RStan. So, it would be a challenge to get Rcpp-specific code in there. Nor does it explain why the hack of calling install_name_tool worked (with clang4 at least). Or why it may not be a problem with clang7 unless you also have Catalina.

jroon · October 10, 2019, 8:36pm

Guys is this of any relevance here: https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features

coatless · October 11, 2019, 1:01am

Unfortunately, no. The issue is Apple stopped shipping an installer package for headers that R uses, which compiles with the fact that R is still looking for the headers in the old location.

torkar · October 11, 2019, 10:09am

Just an FYI: Using Catalina and everything installed according to @coatless’s instructions, if I run @bgoodri’s example or @coatless’s examples it doesn’t matter. I get the same error msg:

Error in throw_exception() : c++ exception (unknown reason)

AlejandroCatalina · October 11, 2019, 11:27am

Again, posting for reference.

I can run simple models it seems, like the examples here. But whenever I try to run rstanrm or a brms I get:

fit ← stan_glm(y ~ x1, data=dat)

SAMPLING FOR MODEL ‘continuous’ NOW (CHAIN 1).
Chain 1: empty_nested() must be true before calling recover_memory()
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " empty_nested() must be true before calling recover_memory()”
error occurred during calling the sampler; sampling not done
Error in check_stanfit(stanfit) :
Invalid stanfit object produced please report bug

or

brm(y ~ x1, data=dat)
Compiling the C++ model
recompiling to avoid crashing R session
Start sampling
SAMPLING FOR MODEL ‘b3e140f1aba3e9c1258901076a1ed29b’ NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 2.5e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.25 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)”
error occurred during calling the sampler; sampling not done
Family: gaussian
Links: mu = identity; sigma = identity
Formula: y ~ x1
Data: dat (Number of observations: 30)
The model does not contain posterior samples.

Thanks!

stevebronder · October 11, 2019, 11:31am

It looks like this is related to the Qs around this email chain. The next few messages talk about diagnosing is the problem is different libc++ versions

https://stat.ethz.ch/pipermail/r-sig-mac/2017-September/012491.html

torkar · October 12, 2019, 3:47pm

Just a question: Do you want me to file this as an Issue on GitHub?

bgoodri · October 12, 2019, 6:17pm

No

bgoodri · October 12, 2019, 8:44pm

If you have Catalina and rstanarm yields any errors when you try to estimate a model. Try installing / testing it from source with

remotes::install_github("stan-dev/rstanarm", build_vignettes = FALSE)
library(rstanarm)
example(example_model)

plus whatever model was not working before.

bgoodri · October 12, 2019, 8:56pm

If you have Catalina and Stan models are not working, try running this function on the object produced by stan_model:

fix_Mac <- function(sm) {
  stopifnot(is(sm, "stanmodel"))
  dso_last_path <- sm@dso@.CXXDSOMISC$dso_last_path
  CLANG_DIR <- tail(n = 1, grep("clang[456789]", 
                    x = list.dirs("/usr/local", recursive = FALSE), 
                    value = TRUE))
  if (length(CLANG_DIR) == 0L) stop("no clang from CRAN found")
  LIBCPP <- file.path(CLANG_DIR, "lib", "libc++.1.dylib")
  if (!file.exists(LIBCPP)) stop("no unique libc++.1?.dylib found")
  Rv <- R.version
  GOOD <- file.path("/Library", "Frameworks", "R.framework", "Versions", 
                     paste(Rv$major, substr(Rv$minor, 1, 1), sep = "."), 
                     "Resources", "lib", "libc++.1.dylib")
  if (!file.exists(GOOD)) stop(paste(GOOD, "not found"))
  cmd <- paste(
    "install_name_tool",
    "-change",
    LIBCPP,
    GOOD,
    dso_last_path
  )
  system(cmd)
  dyn.unload(dso_last_path)
  dyn.load(dso_last_path)
  return(invisible(NULL))
}

followed by sampling(...).

coatless · October 12, 2019, 9:55pm

I’m curious for those who are having difficulties… What happens if you type in Terminal:

R RHOME

Are you getting…

/Library/Frameworks/R.framework/Resources

or something with anaconda? If the later, please avoid using R with anaconda.

torkar · October 13, 2019, 7:19am

So, if I do this:

library(rstan)
library(brms)
output ← c(0.5237868884927709,0.9487179487179488,0.5494563370710159,0.46386348238482383,0.4458125179443009,0.9830952380952381,0.9685185185185186,0.5118241235888294,0.8824457593688363,0.5644325518178729,0.6317411924119242,0.4893159632500718,0.9416071428571429,0.9145833333333333,0.4509011685482273,0.7682445759368836,0.2748216106014271,0.7207063008130081,0.6109249005373036,0.9156547619047619,0.9011574074074074,0.6936423054070112,0.9012820512820513,0.8805895344886171,0.7585789295392954,0.6982066978384808,0.9988095238095238,0.9606481481481481,0.7225985343632402,0.9347140039447732,0.7936459395174992,0.7401422764227643,0.7547029449161232,0.9897619047619047,0.9571759259259259,0.734383046147752,1.0,0.8722222222222221,0.6707232384823848,0.6119692588491038,1.0,1.0,0.5487621311150723,0.8960552268244577,0.6401885830784912,0.7439871273712737,0.6910643533899349,0.9875,0.9847222222222223,0.5955040602099425,0.9396449704142013,0.5083588175331294,0.6947069783197831,0.5883210286698658,0.9872023809523809,0.9643518518518519)
categs<-as.factor(rep(c(“FOO1”,“FOO2”,“FOO3”,“FOO4”,“FOO5”,“FOO6”,“FOO7”,“FOO8”),each=7));
blocks ← c(1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7,1,2,3,4,5,6,7)
df ← data.frame(y=output, algorithm=as.integer(categs), block_id=blocks)

And then

df ← make_standata(y ~ 1 + (1 | algorithm) + (1 | block_id), data = df)

And run model.stan (2.1 KB)

fit ← stan(file = ‘model.stan’, data = df)

I get a c++ exception (unknown reason) during sampling. But…

foo ← stan_model(‘model.stan’)

fix_Mac(foo)

returns nothing, so I guess ok?

sampling(foo, data=df)

Samples nicely.

Just tell me if I misunderstood something… I am somewhat confused but I guess the important thing here is that you’re not confused @bgoodri… ;)

janhannig · October 14, 2019, 1:36pm

I tried it and got:

SAMPLING FOR MODEL ‘pareto_gfi’ NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 7.8e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.78 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
Chain 1: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 1: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 1: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 1: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 1: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 1: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 1: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 1: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 1: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 1: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 1: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 1: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 1:
Chain 1: Elapsed Time: 0.292604 seconds (Warm-up)
Chain 1: 0.285105 seconds (Sampling)
Chain 1: 0.577709 seconds (Total)
Chain 1:

SAMPLING FOR MODEL ‘pareto_gfi’ NOW (CHAIN 2).
Chain 2:
Chain 2: Gradient evaluation took 4.4e-05 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.44 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2:
Chain 2:
Chain 2: Iteration: 1 / 2000 [ 0%] (Warmup)
Chain 2: Iteration: 200 / 2000 [ 10%] (Warmup)
Chain 2: Iteration: 400 / 2000 [ 20%] (Warmup)
Chain 2: Iteration: 600 / 2000 [ 30%] (Warmup)
Chain 2: Iteration: 800 / 2000 [ 40%] (Warmup)
Chain 2: Iteration: 1000 / 2000 [ 50%] (Warmup)
Chain 2: Iteration: 1001 / 2000 [ 50%] (Sampling)
Chain 2: Iteration: 1200 / 2000 [ 60%] (Sampling)
Chain 2: Iteration: 1400 / 2000 [ 70%] (Sampling)
Chain 2: Iteration: 1600 / 2000 [ 80%] (Sampling)
Chain 2: Iteration: 1800 / 2000 [ 90%] (Sampling)
Chain 2: Iteration: 2000 / 2000 [100%] (Sampling)
Chain 2:
Chain 2: Elapsed Time: 0.295372 seconds (Warm-up)
Chain 2: 0.290389 seconds (Sampling)
Chain 2: 0.585761 seconds (Total)
Chain 2:

SAMPLING FOR MODEL ‘pareto_gfi’ NOW (CHAIN 3).
Chain 3:
Chain 3: Gradient evaluation took 4.5e-05 seconds
Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.45 seconds.
Chain 3: Adjust your expectations accordingly!
Chain 3:
Chain 3:
Chain 3: Iteration: 1 / 2000 [ 0%] (Warmup)
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)”
error occurred during calling the sampler; sampling not done
Stan model ‘pareto_gfi’ does not contain samples.

SAMPLING FOR MODEL ‘pareto_gfi’ NOW (CHAIN 1).
Chain 1: empty_nested() must be true before calling recover_memory()
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " empty_nested() must be true before calling recover_memory()”
error occurred during calling the sampler; sampling not done
Stan model ‘pareto_gfi’ does not contain samples.

SAMPLING FOR MODEL ‘pareto_gfi’ NOW (CHAIN 1).
Chain 1: empty_nested() must be true before calling recover_memory()
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " empty_nested() must be true before calling recover_memory()”
error occurred during calling the sampler; sampling not done
Stan model ‘pareto_gfi’ does not contain samples.
Error in { :
task 1 failed - “non-numeric argument to mathematical function”

This is different than before - it crashed on 3rd chain instead of the usual 2nd.

bgoodri · October 14, 2019, 1:38pm

I wouldn’t read too much into that because when exceptions are thrown is random. There could have not been any on the first two chains.

bgoodri · October 14, 2019, 1:45pm

bgoodri:

Can anyone with clang7 or clang8 but not Catalina run this

Rcpp::sourceCpp(code = 
'
#include <Rcpp.h>
using namespace Rcpp; 

// [[Rcpp::export]]
int throw_exception() { 
  std::stringstream errmsg; errmsg << "this is an exception";
  throw std::domain_error(errmsg.str()); 
  return 0;
}
'
)

throw_exception()

to see if you get the “(unknown reason)” thing?

How about doing the above using Xcode’s clang++ and headers? You probably need to comment out everything in ~/.R/Makevars .

coatless · October 14, 2019, 4:15pm

And the entry in ~/.Renviron governing the PATH.

Topic		Replies	Views
Update MacOS to Catalina or Mojave for least problems with Stan-based software? RStan installation	20	1748	December 13, 2020
Installing RStan on Mojave Mac OS X -missing step on C++ toolchain General	4	1030	February 20, 2019
Dealing with Mojave RStan rstan	5	2814	October 7, 2019
MacOS RStan	10	1939	November 2, 2022
Problem with C++ Compiler on Mac RStan	2	2389	October 27, 2018

Dealing with Catalina

Related topics