New Error in RStan

kartermycroft · March 11, 2021, 10:28pm

Hi all,

I posted about a bug I was having with Rstan recently (the infamous ‘cannot allocate vector’ error) and received very helpful responses here which solved the problem. Unfortunately, I’ve just come across a new error that has me completely stumped.

I am running a series of fairly simple Poisson and Negative Binomial models, which haven’t given me any trouble before until I changed some file names and reorganized my project folder a bit (at least, I think that’s what might have precipitated the issue?) Now, whenever I try to run the models, I get the following very long error message:

‘C:/RBUILD~1/4.0/usr/mingw_/bin/g++’ not found
make cmd is
make -f “C:/PROGRA~1/R/R-40~1.1/etc/x64/Makeconf” -f “C:/PROGRA~1/R/R-40~1.1/share/make/winshlib.mk” CXX=‘(CXX14) (CXX14STD)’ CXXFLAGS=‘(CXX14FLAGS)' CXXPICFLAGS='(CXX14PICFLAGS)’ SHLIB_LDFLAGS=‘(SHLIB_CXX14LDFLAGS)' SHLIB_LD='(SHLIB_CXX14LD)’ SHLIB=“file1d5839b933b9.dll” WIN=64 TCLBIN=64 OBJECTS=“file1d5839b933b9.o”
make would use
if test “zfile1d5839b933b9.o” != “z”; then
if test -e “file1d5839b933b9-win.def”; then
echo “C:/RBuildTools/4.0/mingw64/bin/“g++ -shared -s -static-libgcc -o file1d5839b933b9.dll file1d5839b933b9-win.def file1d5839b933b9.o “C:/Users/Karter/Documents/R/win-library/4.0/rstan/lib/x64/libStanServices.a” -L"C:/Users/Karter/Documents/R/win-library/4.0/StanHeaders/libs/x64” -lStanHeaders -L"C:/Users/Karter/Documents/R/win-library/4.0/RcppParallel/lib/x64” -ltbb -L"C:/PROGRA~1/R/R-40~1.1/bin/x64" -lR ;
“C:/RBuildTools/4.0/mingw64/bin/“g++ -shared -s -static-libgcc -o file1d5839b933b9.dll file1d5839b933b9-win.def file1d5839b933b9.o “C:/Users/Karter/Documents/R/win-library/4.0/rstan/lib/x64/libStanServices.a” -L"C:/Users/Karter/Documents/R/win-library/4.0/StanHeaders/libs/x64” -lStanHeaders -L"C:/Users/Karter/Documents/R/win-library/4.0/RcppParallel/lib/x64” -ltbb -L"C:/PROGRA~1/R/R-40~1.1/bin/x64" -lR ;
else
echo EXPORTS > tmp.def;
“C:/RBuildTools/4.0/mingw64/bin/“nm file1d5839b933b9.o | sed -n ‘s/^.* [BCDRT] / /p’ | sed -e ‘/[.]refptr[.]/d’ -e ‘/[.]weak[.]/d’ | sed 's/[^ ][^ ]*/”&”/g’ >> tmp.def;
echo “C:/RBuildTools/4.0/mingw64/bin/“g++ -shared -s -static-libgcc -o file1d5839b933b9.dll tmp.def file1d5839b933b9.o “C:/Users/Karter/Documents/R/win-library/4.0/rstan/lib/x64/libStanServices.a” -L"C:/Users/Karter/Documents/R/win-library/4.0/StanHeaders/libs/x64” -lStanHeaders -L"C:/Users/Karter/Documents/R/win-library/4.0/RcppParallel/lib/x64” -ltbb -L"C:/PROGRA~1/R/R-40~1.1/bin/x64" -lR ;
“C:/RBuildTools/4.0/mingw64/bin/“g++ -shared -s -static-libgcc -o file1d5839b933b9.dll tmp.def file1d5839b933b9.o “C:/Users/Karter/Documents/R/win-library/4.0/rstan/lib/x64/libStanServices.a” -L"C:/Users/Karter/Documents/R/win-library/4.0/StanHeaders/libs/x64” -lStanHeaders -L"C:/Users/Karter/Documents/R/win-library/4.0/RcppParallel/lib/x64” -ltbb -L"C:/PROGRA~1/R/R-40~1.1/bin/x64" -lR ;
rm -f tmp.def;
fi
fi
Error in compileCode(f, code, language = language, verbose = verbose) :
C:/RBuildTools/4.0/mingw64/bin/…/lib/gcc/x86_64-w64-mingw32/8.3.0/…/…/…/…/x86_64-w64-mingw32/bin/ld.exe: C:/Users/Karter/Documents/R/win-library/4.0/rstan/lib/x64/libStanServices.a(stan_fit.o):stan_fit.cpp:(.rdata$_ZZN5boost4math6detail9bessel_j0IeEET_S3_E2P1[_ZZN5boost4math6detail9bessel_j0IeEET_S3_E2P1]+0x0): multiple definition of boost::math::detail::bessel_j0<long double>(long double)::P1'; file1d5839b933b9.o:file1d5839b933b9.cpp:(.data$_ZZN5boost4math6detail9bessel_j0IeEET_S3_E2P1[_ZZN5boost4math6detail9bessel_j0IeEET_S3_E2P1]+0x0): first defined hereC:/RBuildTools/4.0/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/Users/Karter/Documents/R/win-library/4.0/rstan/lib/x64/libStanServices.a(stan_fit.o):stan_fit.cpp:(.rdata$_ZZN5boost4math6detail9bessel_j0IeEET_S3_E2QC[_ZZN5boost4math6detail9bessel_j0IeEET_S3_E2QC]+0x0): multiple definition of boost::math::detail::bessel_j0(long double)::QC’; file1d5839b933b9
Error in sink(type = “output”) : invalid connection

I have tried previous solutions (deleting .rds files, reinstalling everything, writing PATH=“{RTOOLS40_HOME}\usr\bin;{PATH}” to .Renviron) but nothing seems to be working. I’m not able to tell what the error message is telling me except that it might have something to do with file pathing? I also moved all my project files over to a different PC, thinking it might be an antivirus or firewall acting up on my work computer, but I am getting the exact same problem. Initially the problem seemed to only occur with one model, but now it affects anything I run in Rstan including the test formula.

Operating System: Windows 10 Enterprise 64-bit
RStan Version: 2.21.2

Output of writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))

Error in file(con, “r”) : cannot open the connection
In addition: Warning message:
In file(con, “r”) :
cannot open file ‘C:\Users\Karter\Documents/.R/Makevars’: No such file or directory

Output of devtools::session_info("rstan")

Error in get(genname, envir = envir) : object ‘testthat_print’ not found

Session info -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
setting value
version R version 4.0.1 (2020-06-06)
os Windows 10 x64
system x86_64, mingw32
ui RStudio
language (EN)
collate English_United States.1252
ctype English_United States.1252
tz America/Los_Angeles
date 2021-03-11
Packages ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
! package * version date lib source
assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0)
backports 1.1.7 2020-05-13 [1] CRAN (R 4.0.0)
BH 1.72.0-3 2020-01-08 [1] CRAN (R 4.0.0)
callr 3.5.1 2020-10-13 [1] CRAN (R 4.0.4)
checkmate 2.0.0 2020-02-06 [1] CRAN (R 4.0.3)
cli 2.0.2 2020-02-28 [1] CRAN (R 4.0.0)
colorspace 1.4-1 2019-03-18 [1] CRAN (R 4.0.0)
crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.0)
curl 4.3 2019-12-02 [1] CRAN (R 4.0.0)
desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.0)
digest 0.6.25 2020-02-23 [1] CRAN (R 4.0.0)
ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.0)
evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.0)
farver 2.0.3 2020-01-16 [1] CRAN (R 4.0.0)
ggplot2 * 3.3.1 2020-05-28 [1] CRAN (R 4.0.0)
glue 1.4.1 2020-05-13 [1] CRAN (R 4.0.0)
gridExtra 2.3 2017-09-09 [1] CRAN (R 4.0.2)
gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0)
inline 0.3.17 2020-12-01 [1] CRAN (R 4.0.3)
isoband 0.2.1 2020-04-12 [1] CRAN (R 4.0.0)
jsonlite 1.7.2 2020-12-09 [1] CRAN (R 4.0.4)
labeling 0.3 2014-08-23 [1] CRAN (R 4.0.0)
lattice 0.20-41 2020-04-02 [2] CRAN (R 4.0.1)
lifecycle 0.2.0 2020-03-06 [1] CRAN (R 4.0.0)
loo 2.4.1 2020-12-09 [1] CRAN (R 4.0.3)
magrittr 1.5 2014-11-22 [1] CRAN (R 4.0.0)
MASS 7.3-51.6 2020-04-26 [2] CRAN (R 4.0.1)
Matrix 1.2-18 2019-11-27 [2] CRAN (R 4.0.1)
matrixStats 0.57.0 2020-09-25 [1] CRAN (R 4.0.3)
mgcv 1.8-31 2019-11-09 [2] CRAN (R 4.0.1)
munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0)
nlme 3.1-148 2020-05-24 [2] CRAN (R 4.0.1)
pillar 1.4.4 2020-05-05 [1] CRAN (R 4.0.0)
pkgbuild 1.2.0 2020-12-15 [1] CRAN (R 4.0.4)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
pkgload 1.1.0 2020-05-29 [1] CRAN (R 4.0.0)
praise 1.0.0 2015-08-11 [1] CRAN (R 4.0.0)
prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0)
processx 3.4.5 2020-11-30 [1] CRAN (R 4.0.4)
ps 1.3.3 2020-05-08 [1] CRAN (R 4.0.0)
R6 2.4.1 2019-11-12 [1] CRAN (R 4.0.0)
RColorBrewer 1.1-2 2014-12-07 [1] CRAN (R 4.0.0)
Rcpp 1.0.6 2021-01-15 [1] CRAN (R 4.0.3)
RcppEigen 0.3.3.7.0 2019-11-16 [1] CRAN (R 4.0.0)
D RcppParallel 5.0.2 2020-06-24 [1] CRAN (R 4.0.3)
rlang 0.4.10 2020-12-30 [1] CRAN (R 4.0.4)
rprojroot 1.3-2 2018-01-03 [1] CRAN (R 4.0.0)
rstan * 2.21.2 2020-07-27 [1] CRAN (R 4.0.4)
rstudioapi 0.11 2020-02-07 [1] CRAN (R 4.0.0)
scales 1.1.1 2020-05-11 [1] CRAN (R 4.0.0)
StanHeaders * 2.21.0-7 2020-12-17 [1] CRAN (R 4.0.4)
testthat 2.3.2 2020-03-02 [1] CRAN (R 4.0.0)
tibble * 3.0.1 2020-04-20 [1] CRAN (R 4.0.0)
utf8 1.1.4 2018-05-24 [1] CRAN (R 4.0.0)
V8 3.4.0 2020-11-04 [1] CRAN (R 4.0.3)
vctrs 0.3.1 2020-06-05 [1] CRAN (R 4.0.0)
viridisLite 0.3.0 2018-02-01 [1] CRAN (R 4.0.0)
withr 2.4.1 2021-01-26 [1] CRAN (R 4.0.4)

[1] C:/Users/Karter/Documents/R/win-library/4.0
[2] C:/Program Files/R/R-4.0.1/library

D – DLL MD5 mismatch, broken installation.

Thanks all in advance for any help you can give!

andrjohns · March 11, 2021, 10:45pm

Hi Karter,

You just need to reinstall RStan and StanHeaders from source:

# Compile packages using all cores
Sys.setenv(MAKEFLAGS = paste0("-j",parallel::detectCores()))

install.packages(c("StanHeaders","rstan"),type="source")

kartermycroft · March 11, 2021, 11:03pm

Thanks…That seems to help somewhat. I had tried reinstalling from source but didn’t include the line of code you shared beforehand (mind explaining what that one does?)

Now I’m back to the problem I was having originally, which was R crashing anytime I run more than two Stan models. I can’t figure out what the issue is there. It’s not running out of RAM or anything. The first two models I run work just fine but the third one seems to force RStudio to abort every time. I’ve never had this issue before, even when running a huge series of code with dozens of models. So not sure what might be causing it or if it’s related to the error message I was getting.

Let me know if you have any ideas! If the error message doesn’t come back, maybe I should make a new topic for the crashes. Thanks so much!

andrjohns · March 12, 2021, 12:25pm

That first line just instructs R to use all available cores when building packages from source.

The main causes for crashes (generally) are:

Having -march=native in the Makevars.win file
Repeatedly assigning RStan runs to the same output object (i.e. overwriting)

If neither of those are relevant in your case then you could also try the experimental preview of RStan 2.26 (in a new session):

remove.packages(c("StanHeaders", "rstan"))
install.packages("StanHeaders", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

kartermycroft · March 12, 2021, 8:57pm

Brilliant! I don’t have a Makevars.win file (I think I deleted or renamed it as a fix to a previous bug), but addressing the overwriting issue seems to have helped significantly. I was indeed assigning subsequent models to the same object name since they all need to feed into the same prediction functions, but with just a few edits to the syntax I’ve managed to avoid any overwriting when compiling the Stan models and can run all the models without crashes.

To summarize in case anyone has similar issues in the future, here’s what worked:
-Reinstalling RStan and StanHeaders from source, using all cores
-Ensuring that each run of the stan() function is assigned to a unique object.

Thanks so much for the help!!

Topic		Replies	Views
New Error: 'C:/rtools40/usr/mingw_/bin/g++' not found; Allocating Large Vector RStan bug , compiler	6	3699	September 23, 2020
Cannot install Rstan General installation	3	621	September 15, 2021
Rstan install trouble RStan installation	34	1848	December 15, 2020
Error in compilation after rstan reinstall General rstan	2	391	July 17, 2023
I run a stan model with Rstan and i get this Error : cannot allocate vector of size xxx Mb Modeling	1	560	February 8, 2021

New Error in RStan

Related topics