Error compiling/running model in Rstan in Mac OSX Catalina

Hi All,
I’ve been banging away at this problem for two days - basically up until then Rstan was running fine but it suddenly stopped and the errors below started. I was running R 4.0.2, with OSX Catalina 10.15.5, but today I downgraded to R 3.6.3 to try to solve the problem - but I get the same set of errors you see below. I followed the installation on: https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started. When I run a single chain, sampling actually happens, but I still get the same set of errors prior to the sampling seen below.

More system info is below the output.

Any help is appreciated.
Best,
MG

 fit.direct<- stan(file = stan_model,data = stan_data,chains = 2,iter = 2000000,control=list(adapt_delta=0.95))

clang-7: error: no such file or directory: 'MG/Stan'
clang-7: error: no such file or directory: 'Functions'
clang-7: error: no such file or directory: 'MG/Stan'
clang-7: error: no such file or directory: 'Functions'
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
 https://github.com/stan-dev/stanc3/issues 
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
cannot remove file '/var/folders/rc/nmtkkjgj1qggg9jztthftdn00000gn/T//RtmpgyFGrp/file1c6478ca36dd.stan', reason 'No such file or directory'starting worker pid=7451 on localhost:11578 at 21:59:50.683
starting worker pid=7465 on localhost:11578 at 21:59:50.850

SAMPLING FOR MODEL 'bayes_slouch_cc_v3' NOW (CHAIN 1).

SAMPLING FOR MODEL 'bayes_slouch_cc_v3' NOW (CHAIN 2).
[1] "Error : vector memory exhausted (limit reached?)"
error occurred during calling the sampler; sampling not done
[1] "Error : vector memory exhausted (limit reached?)"
error occurred during calling the sampler; sampling not done
some chains had errors; consider specifying chains = 1 to debughere are whatever error messages were returned
[[1]]
Stan model 'bayes_slouch_cc_v3' does not contain samples.

[[2]]
Stan model 'bayes_slouch_cc_v3' does not contain samples.
  • Operating System OSX Catalina 10.15.5

  • RStan Version 2.21.2

  • Output of writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))

clang: start

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,{R_HOME}/lib {R_HOME}/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,{R_HOME}/lib {R_HOME}/lib/libc++abi.1.dylib

clang: end

  • Output of devtools::session_info("rstan")

devtools::session_info(“rstan”)
─ Session info ──────────────────────────────────────────────────────────────
setting value
version R version 3.6.3 (2020-02-29)
os macOS Catalina 10.15.5
system x86_64, darwin15.6.0
ui AQUA
language (EN)
collate en_GB.UTF-8
ctype en_GB.UTF-8
tz Europe/London
date 2020-07-11

─ Packages ──────────────────────────────────────────────────────────────────
package * version date lib source
assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.3)
backports 1.1.8 2020-06-17 [1] CRAN (R 3.6.3)
BH 1.72.0-3 2020-01-08 [1] CRAN (R 3.6.3)
callr 3.4.3 2020-03-28 [1] CRAN (R 3.6.3)
checkmate 2.0.0 2020-02-06 [1] CRAN (R 3.6.3)
cli 2.0.2 2020-02-28 [1] CRAN (R 3.6.3)
colorspace 1.4-1 2019-03-18 [1] CRAN (R 3.6.3)
crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.3)
curl 4.3 2019-12-02 [1] CRAN (R 3.6.3)
desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.3)
digest 0.6.25 2020-02-23 [1] CRAN (R 3.6.3)
ellipsis 0.3.1 2020-05-15 [1] CRAN (R 3.6.3)
evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.3)
fansi 0.4.1 2020-01-08 [1] CRAN (R 3.6.3)
farver 2.0.3 2020-01-16 [1] CRAN (R 3.6.3)
ggplot2 3.3.2 2020-06-19 [1] CRAN (R 3.6.3)
glue 1.4.1 2020-05-13 [1] CRAN (R 3.6.3)
gridExtra 2.3 2017-09-09 [1] CRAN (R 3.6.3)
gtable 0.3.0 2019-03-25 [1] CRAN (R 3.6.3)
inline 0.3.15 2018-05-18 [1] CRAN (R 3.6.3)
isoband 0.2.2 2020-06-20 [1] CRAN (R 3.6.3)
jsonlite 1.7.0 2020-06-25 [1] CRAN (R 3.6.3)
labeling 0.3 2014-08-23 [1] CRAN (R 3.6.3)
lattice 0.20-41 2020-04-02 [1] CRAN (R 3.6.2)
lifecycle 0.2.0 2020-03-06 [1] CRAN (R 3.6.3)
loo 2.3.0 2020-07-07 [1] CRAN (R 3.6.2)
magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.3)
MASS 7.3-51.6 2020-04-26 [1] CRAN (R 3.6.2)
Matrix 1.2-18 2019-11-27 [1] CRAN (R 3.6.3)
matrixStats 0.56.0 2020-03-13 [1] CRAN (R 3.6.3)
mgcv 1.8-31 2019-11-09 [1] CRAN (R 3.6.3)
munsell 0.5.0 2018-06-12 [1] CRAN (R 3.6.3)
nlme 3.1-148 2020-05-24 [1] CRAN (R 3.6.2)
pillar 1.4.6 2020-07-10 [1] CRAN (R 3.6.3)
pkgbuild 1.0.8 2020-05-07 [1] CRAN (R 3.6.3)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.6.3)
pkgload 1.1.0 2020-05-29 [1] CRAN (R 3.6.3)
praise 1.0.0 2015-08-11 [1] CRAN (R 3.6.3)
prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.6.3)
processx 3.4.3 2020-07-05 [1] CRAN (R 3.6.3)
ps 1.3.3 2020-05-08 [1] CRAN (R 3.6.3)
R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.3)
RColorBrewer 1.1-2 2014-12-07 [1] CRAN (R 3.6.3)
Rcpp 1.0.5.1 2020-07-11 [1] local
RcppEigen 0.3.3.7.0 2019-11-16 [1] CRAN (R 3.6.3)
RcppParallel 5.0.2 2020-06-24 [1] CRAN (R 3.6.3)
rlang 0.4.7 2020-07-09 [1] CRAN (R 3.6.3)
rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.3)
rstan 2.21.2 2020-07-11 [1] Github (stan-dev/rstan@a9b5554)
rstudioapi 0.11 2020-02-07 [1] CRAN (R 3.6.3)
scales 1.1.1 2020-05-11 [1] CRAN (R 3.6.3)
StanHeaders 2.21.0-5 2020-06-09 [1] CRAN (R 3.6.3)
testthat 2.3.2 2020-03-02 [1] CRAN (R 3.6.3)
tibble 3.0.3 2020-07-10 [1] CRAN (R 3.6.3)
utf8 1.1.4 2018-05-24 [1] CRAN (R 3.6.3)
V8 3.2.0 2020-06-19 [1] CRAN (R 3.6.2)
vctrs 0.3.1 2020-06-05 [1] CRAN (R 3.6.3)
viridisLite 0.3.0 2018-02-01 [1] CRAN (R 3.6.3)
withr 2.2.0 2020-04-20 [1] CRAN (R 3.6.3)

[1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library

If you

remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan")

the message about

The NEXT version of Stan will not be able to pre-process your Stan program.

will probably go away, but I don’t know about the

vector memory exhausted (limit reached?)

@wds15 This sounds like a Mac version of stack overflow on Windows.

Ok…I can try to reproduce on Mac then.

Edit: why do you ever need 2 Million samples??? Can you try with fewer samples please…like 2k?

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

So, you do get your result?

What does look odd to me is that you are using apparently clang-7 which suggests that you do not use clang from XCode - which is what you should use with R 4.0 onwards.

I think the errors you see are related to changes wrt to mclappy introduced by @bgoodri. These are already under consideration. You should be able to avoid them with setting cores=1 as argument to the stan call in order to sample with just one core.

@bgoodri I think we are back to windows only problems. This Mac memory issue was due to an extreme attempt to sample 2* 10^6 iterations.

OK. Although the Windows thing seems to come up on some machines with just the 8 schools example from the wiki.

Hi. It does get the result I expect, thanks. I still get error messages though - this morning I re-upgraded to R 4.0.2 and went through the setup process again. Getting consistent results, but still getting error messages related to clang, though slightly different - see below.
Thanks,
MG

clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
https://github.com/stan-dev/stanc3/issues
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
cannot remove file ‘/var/folders/rc/nmtkkjgj1qggg9jztthftdn00000gn/T//RtmpukzweW/filea8775068641.stan’, reason 'No such file or directory

I believe those messages will go away if you temporarily do

remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan")

Thanks - I still get the same set of errors, but like I said, everything seems to be running fine.
Best,
MG

DIAGNOSTIC(S) FROM PARSER:

clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
https://github.com/stan-dev/stanc3/issues
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
cannot remove file ‘/var/folders/rc/nmtkkjgj1qggg9jztthftdn00000gn/T//Rtmps9HIzr/file8e6e7ab01148.stan’, reason 'No such file or directory’hash mismatch so recompiling; make sure Stan code ends with a blank line

Don’t worry too much. This is fixed on GitHub and the fix will be in the next StanHeaders for CRAN.