Error compiling/running model in Rstan in Mac OSX Catalina

mgrab · July 11, 2020, 9:19pm

Hi All,
I’ve been banging away at this problem for two days - basically up until then Rstan was running fine but it suddenly stopped and the errors below started. I was running R 4.0.2, with OSX Catalina 10.15.5, but today I downgraded to R 3.6.3 to try to solve the problem - but I get the same set of errors you see below. I followed the installation on: RStan Getting Started · stan-dev/rstan Wiki · GitHub. When I run a single chain, sampling actually happens, but I still get the same set of errors prior to the sampling seen below.

More system info is below the output.

Any help is appreciated.
Best,
MG

 fit.direct<- stan(file = stan_model,data = stan_data,chains = 2,iter = 2000000,control=list(adapt_delta=0.95))

clang-7: error: no such file or directory: 'MG/Stan'
clang-7: error: no such file or directory: 'Functions'
clang-7: error: no such file or directory: 'MG/Stan'
clang-7: error: no such file or directory: 'Functions'
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
 https://github.com/stan-dev/stanc3/issues 
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
cannot remove file '/var/folders/rc/nmtkkjgj1qggg9jztthftdn00000gn/T//RtmpgyFGrp/file1c6478ca36dd.stan', reason 'No such file or directory'starting worker pid=7451 on localhost:11578 at 21:59:50.683
starting worker pid=7465 on localhost:11578 at 21:59:50.850

SAMPLING FOR MODEL 'bayes_slouch_cc_v3' NOW (CHAIN 1).

SAMPLING FOR MODEL 'bayes_slouch_cc_v3' NOW (CHAIN 2).
[1] "Error : vector memory exhausted (limit reached?)"
error occurred during calling the sampler; sampling not done
[1] "Error : vector memory exhausted (limit reached?)"
error occurred during calling the sampler; sampling not done
some chains had errors; consider specifying chains = 1 to debughere are whatever error messages were returned
[[1]]
Stan model 'bayes_slouch_cc_v3' does not contain samples.

[[2]]
Stan model 'bayes_slouch_cc_v3' does not contain samples.

Operating System OSX Catalina 10.15.5
RStan Version 2.21.2
Output of writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))

clang: start

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include

SHLIB_CXXLDFLAGS+=-Wl,-rpath,{R_HOME}/lib {R_HOME}/lib/libc++abi.1.dylib
SHLIB_CXX14LDFLAGS+=-Wl,-rpath,{R_HOME}/lib {R_HOME}/lib/libc++abi.1.dylib

clang: end

Output of devtools::session_info("rstan")

devtools::session_info(“rstan”)
─ Session info ──────────────────────────────────────────────────────────────
setting value
version R version 3.6.3 (2020-02-29)
os macOS Catalina 10.15.5
system x86_64, darwin15.6.0
ui AQUA
language (EN)
collate en_GB.UTF-8
ctype en_GB.UTF-8
tz Europe/London
date 2020-07-11

─ Packages ──────────────────────────────────────────────────────────────────
package * version date lib source
assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.3)
backports 1.1.8 2020-06-17 [1] CRAN (R 3.6.3)
BH 1.72.0-3 2020-01-08 [1] CRAN (R 3.6.3)
callr 3.4.3 2020-03-28 [1] CRAN (R 3.6.3)
checkmate 2.0.0 2020-02-06 [1] CRAN (R 3.6.3)
cli 2.0.2 2020-02-28 [1] CRAN (R 3.6.3)
colorspace 1.4-1 2019-03-18 [1] CRAN (R 3.6.3)
crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.3)
curl 4.3 2019-12-02 [1] CRAN (R 3.6.3)
desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.3)
digest 0.6.25 2020-02-23 [1] CRAN (R 3.6.3)
ellipsis 0.3.1 2020-05-15 [1] CRAN (R 3.6.3)
evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.3)
fansi 0.4.1 2020-01-08 [1] CRAN (R 3.6.3)
farver 2.0.3 2020-01-16 [1] CRAN (R 3.6.3)
ggplot2 3.3.2 2020-06-19 [1] CRAN (R 3.6.3)
glue 1.4.1 2020-05-13 [1] CRAN (R 3.6.3)
gridExtra 2.3 2017-09-09 [1] CRAN (R 3.6.3)
gtable 0.3.0 2019-03-25 [1] CRAN (R 3.6.3)
inline 0.3.15 2018-05-18 [1] CRAN (R 3.6.3)
isoband 0.2.2 2020-06-20 [1] CRAN (R 3.6.3)
jsonlite 1.7.0 2020-06-25 [1] CRAN (R 3.6.3)
labeling 0.3 2014-08-23 [1] CRAN (R 3.6.3)
lattice 0.20-41 2020-04-02 [1] CRAN (R 3.6.2)
lifecycle 0.2.0 2020-03-06 [1] CRAN (R 3.6.3)
loo 2.3.0 2020-07-07 [1] CRAN (R 3.6.2)
magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.3)
MASS 7.3-51.6 2020-04-26 [1] CRAN (R 3.6.2)
Matrix 1.2-18 2019-11-27 [1] CRAN (R 3.6.3)
matrixStats 0.56.0 2020-03-13 [1] CRAN (R 3.6.3)
mgcv 1.8-31 2019-11-09 [1] CRAN (R 3.6.3)
munsell 0.5.0 2018-06-12 [1] CRAN (R 3.6.3)
nlme 3.1-148 2020-05-24 [1] CRAN (R 3.6.2)
pillar 1.4.6 2020-07-10 [1] CRAN (R 3.6.3)
pkgbuild 1.0.8 2020-05-07 [1] CRAN (R 3.6.3)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.6.3)
pkgload 1.1.0 2020-05-29 [1] CRAN (R 3.6.3)
praise 1.0.0 2015-08-11 [1] CRAN (R 3.6.3)
prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.6.3)
processx 3.4.3 2020-07-05 [1] CRAN (R 3.6.3)
ps 1.3.3 2020-05-08 [1] CRAN (R 3.6.3)
R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.3)
RColorBrewer 1.1-2 2014-12-07 [1] CRAN (R 3.6.3)
Rcpp 1.0.5.1 2020-07-11 [1] local
RcppEigen 0.3.3.7.0 2019-11-16 [1] CRAN (R 3.6.3)
RcppParallel 5.0.2 2020-06-24 [1] CRAN (R 3.6.3)
rlang 0.4.7 2020-07-09 [1] CRAN (R 3.6.3)
rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.3)
rstan 2.21.2 2020-07-11 [1] Github (stan-dev/rstan@a9b5554)
rstudioapi 0.11 2020-02-07 [1] CRAN (R 3.6.3)
scales 1.1.1 2020-05-11 [1] CRAN (R 3.6.3)
StanHeaders 2.21.0-5 2020-06-09 [1] CRAN (R 3.6.3)
testthat 2.3.2 2020-03-02 [1] CRAN (R 3.6.3)
tibble 3.0.3 2020-07-10 [1] CRAN (R 3.6.3)
utf8 1.1.4 2018-05-24 [1] CRAN (R 3.6.3)
V8 3.2.0 2020-06-19 [1] CRAN (R 3.6.2)
vctrs 0.3.1 2020-06-05 [1] CRAN (R 3.6.3)
viridisLite 0.3.0 2018-02-01 [1] CRAN (R 3.6.3)
withr 2.2.0 2020-04-20 [1] CRAN (R 3.6.3)

[1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library

bgoodri · July 11, 2020, 9:26pm

If you

remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan")

the message about

The NEXT version of Stan will not be able to pre-process your Stan program.

will probably go away, but I don’t know about the

vector memory exhausted (limit reached?)

@wds15 This sounds like a Mac version of stack overflow on Windows.

wds15 · July 11, 2020, 9:33pm

Ok…I can try to reproduce on Mac then.

Edit: why do you ever need 2 Million samples??? Can you try with fewer samples please…like 2k?

mgrab · July 12, 2020, 7:08am

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

wds15 · July 12, 2020, 7:40am

So, you do get your result?

What does look odd to me is that you are using apparently clang-7 which suggests that you do not use clang from XCode - which is what you should use with R 4.0 onwards.

I think the errors you see are related to changes wrt to mclappy introduced by @bgoodri. These are already under consideration. You should be able to avoid them with setting cores=1 as argument to the stan call in order to sample with just one core.

@bgoodri I think we are back to windows only problems. This Mac memory issue was due to an extreme attempt to sample 2* 10^6 iterations.

bgoodri · July 12, 2020, 1:08pm

OK. Although the Windows thing seems to come up on some machines with just the 8 schools example from the wiki.

mgrab · July 12, 2020, 4:31pm

Hi. It does get the result I expect, thanks. I still get error messages though - this morning I re-upgraded to R 4.0.2 and went through the setup process again. Getting consistent results, but still getting error messages related to clang, though slightly different - see below.
Thanks,
MG

clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
https://github.com/stan-dev/stanc3/issues
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
cannot remove file ‘/var/folders/rc/nmtkkjgj1qggg9jztthftdn00000gn/T//RtmpukzweW/filea8775068641.stan’, reason 'No such file or directory

bgoodri · July 12, 2020, 5:13pm

I believe those messages will go away if you temporarily do

remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan")

mgrab · July 14, 2020, 3:02pm

Thanks - I still get the same set of errors, but like I said, everything seems to be running fine.
Best,
MG

DIAGNOSTIC(S) FROM PARSER:

clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
clang: error: no such file or directory: ‘MG/Stan’
clang: error: no such file or directory: ‘Functions’
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
https://github.com/stan-dev/stanc3/issues
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
cannot remove file ‘/var/folders/rc/nmtkkjgj1qggg9jztthftdn00000gn/T//Rtmps9HIzr/file8e6e7ab01148.stan’, reason 'No such file or directory’hash mismatch so recompiling; make sure Stan code ends with a blank line

bgoodri · July 15, 2020, 2:39am

Don’t worry too much. This is fixed on GitHub and the fix will be in the next StanHeaders for CRAN.

Topic		Replies	Views
Compilation error (rstan) General	2	1062	July 3, 2019
Error when running Rstan with update RStan	28	3247	March 28, 2020
Model_header.hpp:4:25: fatal error: stan/math.hpp: No such file or directory RStan	6	2030	February 4, 2019
Stan stopped working RStan rstan	14	1136	April 19, 2021
Stan files won't compile since making changes related to Catalina RStan	3	897	December 22, 2019

Error compiling/running model in Rstan in Mac OSX Catalina

clang: start

clang: end

Related topics