Dealing with Catalina IV

I’m getting weird behavior on Catalina when compiling any model that has an error. Instead of reporting the error, stan_model or stanc just returns with no error message, and no value returned. Thereafter no error messages nor warning messages of any sort are reported by any other command executed. Digging deeper I found that the FFI call within stanc fails, throwing an unknown C++ exception, and fails to properly restore stderr (which it had redirected to a file). I’ve filed this as issue #751 for standev/rstan.

Despite this, I can compile and run the 8schools example without problem, and likewise I can run prophet, which relies on RStan.

Yes, I carefully followed the instructions for installing RStan on MacOS Catalina using the “macOS R toolchain installer” linked to on the page “Installing RStan from source on a Mac”, and this was fairly recently, with the last few weeks.

Has anyone else encountered this problem? Does anyone know of a reasonable work-around? It’s difficult to debug a Stan model if you can’t get back any error messages from the compiler.

When stanc returns nothing that just means there was a syntax error but Catalina is causing the error message to get lost. If you install rstan from source via CRAN, it should resolve itself.

Thanks, that seems to solve it. It’s puzzling, though, because I DID originally install RStan from source via CRAN, just a few weeks ago, and it was the same version (2.19.3).

@bgoodri sorry to bother you again but just curious if there was any other suggestion to try besides going in and editing the command line tools as suggested by @Tan_Phan? Mac pushed out a command line tools update a couple of days ago but that didn’t seem to solve eh problem…

@coatless: Have you seen errors like the one @davidh is encountering that look like

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/unistd.h:727:31: error: unknown type name ‘uuid_t’; did you mean ‘uid_t’?

@bgoodri there is an issue with the current version of Rcpp v1.0.4 on CRAN.

In short, it’s now required to compile Rcpp from source at its drat repository

The following should be used until v1.0.5 or R 4.0.0 is released (on April 24th):

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")
install.packages("rstan", type = "source")

Details on the Rcpp buglet:

Thanks @coatless! For anyone else with this issue, I had to first install Rcpp from

install.packages(“Rcpp”, repos = “https://rcppcore.github.io/drat”)

and then install rstan

Can we add,

install.packages("Rcpp", repos="https://rcppcore.github.io/drat")

to the main rstan install instructions at,

and

until the bug is fixed?

Done

Hey,

I m experiencing the same issues ([1] "Error in sampler$call_sampler(args_list[[i]]) : “[2] " c++ exception (unknown reason)” error occurred during calling the sampler; sampling not done)

I ve been trying to follow the steps outlined here:

  1. Downloaded toolchain - rmacoslib/r-macos-rtools
  2. install.packages(c(“StanHeaders”, “rstan”), type = “source”)```

I have seen posts elsewhere about suggesting replacing Makevars - i m not sure what that is and whether it is necessary given the thread above.

I have Catalina and the newest R version and R-studio.

Thanks

The r-macos-rtools should setup your C++ toolchain correctly (but it seems to not always succeed). If you try

install.packages("rstanarm", type = "source")

the unknown c++ exception messages might go away.

Thanks!!!

Indeed worked!!!

Gabor

Edit: I’m on MacOS 10.15.5 and using R version 4.0.0 (2020-04-24)

@bgoodri @coatless I recently, accidentally updated RStan to 2.21 and Rcpp. I tried to uninstall it and reinstall Rcpp from drat (i.e., install.packages("Rcpp", repos = "https://rcppcore.github.io/drat")) but I now get

Warning in install.packages :
  unable to access index for repository https://rcppcore.github.io/drat/bin/macosx/contrib/4.0:
  cannot open URL 'https://rcppcore.github.io/drat/bin/macosx/contrib/4.0/PACKAGES'

If I just install the current Rcpp and then try to install RStan, I get the following error:

In file included from stan_fit.cpp:33:
In file included from ./stan/services/diagnose/diagnose.hpp:10:
In file included from ./stan/model/test_gradients.hpp:7:
In file included from ./stan/model/log_prob_grad.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:51:
/Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/rev/mat/fun/squared_distance.hpp:27:11: warning: unused type alias 'idx_t' [-Wunused-local-typedef]
    using idx_t = typename index_type<matrix_v>::type;
          ^
/Library/Frameworks/R.framework/Versions/4.0/Resources/library/StanHeaders/include/stan/math/rev/mat/fun/squared_distance.hpp:64:11: warning: unused type alias 'idx_t' [-Wunused-local-typedef]
    using idx_t = typename index_type<matrix_d>::type;
          ^
26 warnings generated.
ar -rs ../inst/lib//libStanServices.a stan_fit.o stan_fit_base.o
ar: creating archive ../inst/lib//libStanServices.a
installing to /Library/Frameworks/R.framework/Versions/4.0/Resources/library/00LOCK-rstan/00new/rstan/libs
** R
** inst
** byte-compile and prepare package for lazy loading
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/libs/Rcpp.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/libs/Rcpp.so, 6): Symbol not found: _EXTPTR_PTR
  Referenced from: /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/libs/Rcpp.so
  Expected in: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libR.dylib
 in /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/libs/Rcpp.so
Calls: <Anonymous> ... asNamespace -> loadNamespace -> library.dynam -> dyn.load
Execution halted
ERROR: lazy loading failed for package ‘rstan’
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/rstan’
Warning in install.packages :
  installation of package ‘rstan’ had non-zero exit status

The downloaded source packages are in
	‘/private/var/folders/zz/dvtr7nq911q1m2m0583x4tv40000gn/T/Rtmp7Oufnp/downloaded_packages’

Any ideas?

I would try installing Rcpp from source (either from CRAN or GitHub) but it is supposed to work if everything is using XCode’s clang++.

Not sure exactly what happened but I just updated to R4.0.2 and everything installed smoothly after that. Thanks for the quick response!

1 Like

Hello,

I’m running Catalina 10.15.6 and R 3.6.1 and have manually installed the toolchain (Xcode CLI, clang7, gfortran6.1) following @coatless instructions. I have also installed rstan , stanheaders and rstanarm from source following @bgoodri advice in previous posts . The install appears fine, the “helloworld.cpp” test runs fine, RStan examples run fine (from e.g. here), everything seems fine - works perfect.

But I’m getting the following random error when trying to run a brm model - has anybody seen it or similar?

>  m<-brm(log_fecundity~log_WL+(1+log_WL|Ref_unique_species), data=dat, chains=3, iter=10000, warmup=5000,
+          prior=prior, control=list(adapt_delta=0.99, max_treedepth=17), thin=5,
+          file='aeAlbFec_ref')
Compiling Stan program...
/var/folders/kh/lyr1fly54gb6m19c19pbm4jh_2gv05/T//RtmpnzLw2K/file9c252f58ef5.stan:42:45: warning: missing terminating ' character [-Winvalid-pp-token]
  r_1 = (diag_pre_multiply(sd_1, L_1) * z_1)';
                                            ^
1 warning generated.

It’s random, as if I shutdown/restart either RStudio or the whole laptop, and try running it again, it will sometimes compile and run (fast - <1 second), without warning.
Sometimes the warning will appear but it will still run through all chains, although take a long, long time to complete (> 10 mins). Other times the warning will appear, the chains will not run and the fan in the laptop starts getting loud, and it appears as though the system has hung.

I’ve tried uninstall/reinstall everything - but each time it appears to install fine, and my models will run without error, but then I’ll start another model, or re-run the same one, and the error starts to appear.

Any ideas? I tried searching for /var/folders… info and “missing terminating ’ character” info but couldn’t find anything. Not too sure what this all means either…

Cheers
Jon

You can ignore that.

Hi,
I have been trying to download Rcpp and Rstan and encountered some of the same errors as davidh.

When downloading Rcpp from drat, I get the following warning in the first few lines:

Warning in install.packages :
  unable to access index for repository https://rcppcore.github.io/drat/bin/macosx/contrib/4.0:
  cannot open URL 'https://rcppcore.github.io/drat/bin/macosx/contrib/4.0/PACKAGES'

Other warning will also pop up throughout the download including a warning about the clang (attached). I’m a little confused about this one as well. I’m on Catalina (10.15), but it says I don’t meet the 10.13+.

1 warning generated. 
clan++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include/  -I/usr/local/include-fPIC -Wall -g -02 attributes.cpp -o attributes.o
In file included from attributes.cpp:39: 

When I download Rstan, several sets of warnings are generated as well. One of which looks like the section below:

30 warnings generated.
ar -rs ../inst/lib//libStanServices.a stan_fit.o stan_fit_base.o
ar: creating archive ../inst/lib//libStanServices.a
installing to /Library/Frameworks/R.framework/Versions/4.0/Resources/library/00LOCK-rstan/00new/rstan/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (rstan

Has anyone come across some different solutions that can help with these issues?

For context, I’m running on Catalina 10.15.6 and R 4.0.2. So, I’m not able to use the same solution as davidh.

With R 4.x, you should be able to install all the binaries from CRAN with no ~/.R/Makevars . On Macs, you might need to eliminate the PATH line from the ~/.Renviron file if you have that from an older version of the C++ toolchain.

Thank you so much for the response! At the time of the first post, I had the latest macOS R toolchain installed, and the ~/.Renviron file was not linked to older versions of clang.

Should I be looking in locations other than the ~/.Renviron file for an out of place PATH line?