Cannot resolve the "cannot allocate vector of size … " error

Following https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started:
6. install.packages(“rstan”, repos = “https://cloud.r-project.org/”, dependencies = TRUE), installed rstan no problem
7.pkgbuild::has_build_tools(debug = TRUE), error message in RStudio: “Building R package from source requires installation of additional build tools. Do you want to install the additionalk tools now?”
Confused as Rtools 4.0 is clearly installed. I checked the Windows apps.
7a: click yes: FALSE
7b: click no: FALSE

In what directory is Rtools installed?

Honestly, I think this might be due to some unnecessary Makevars.win configuration. The wiki is outdated.

Uninstall Rtools (all versions). Install Rtools 40. Rename your Makevars.win to something else, just to remove it from the picture for now. And try again. You may also need to rename .Renviron to something else as well.

First, a default configuration for Rtools 40 should not be C:/Rtools at all. It should be C:/rtools40.

Second, ensure after running Rtools40, that you open R and run: writeLines('PATH="${RTOOLS40_HOME}\\usr\\bin;${PATH}"', con = "~/.Renviron")

Third, if that still does not work, my guess is you have some lingering PATH problems, and a lingering Rtools directory.

You do not need a Makevars.win file with R 4.0 and Rtools40 to run Stan; it’s mainly useful for adding compiler optimizations, which should be done after ensuring that it works. All you should need, is R 4.0, Rtools40 (with defaults selected), and to run the writeLines('PATH="${RTOOLS40_HOME}\\usr\\bin;${PATH}"', con = "~/.Renviron") command in R. Then restart R, and it should compile the model.

Hi,
had the same problem as you since upgarding to R 4.0 and followed the wiki to install Rstan, and amend the Makevars.win files.
Still, I got error messages “cannot allocate…”
Turned out my code was also using the recommendation to avoid recompilation of unchanged Stan programs by calling
rstan::rstan_options(auto_write = TRUE)

However my stanmodel.rds were outdated. Once these were removed and models were recompiled, everything was fine.

Could that be part of your problem?

Cheers

1 Like

I uninstalled everything completely. Then re-installed just R4.0 and rtools40.
Followed your instructions renaming Makevars.win and .Renviron to something else. Same error: “cannot allocate vector of size …” Then did your recommended steps:

  1. rtools installed correctly in C:/rtools40

  2. Console output:
    TRANSLATING MODEL ‘flights_model’ FROM Stan CODE TO C++ CODE NOW.
    successful in parsing the Stan model ‘flights_model’.
    CHECKING DATA AND PREPROCESSING FOR MODEL ‘flights_model’ NOW.
    COMPILING MODEL ‘flights_model’ NOW.
    Error in get(“storage”, envir = as.environment(x)) :
    object ‘storage’ not found

  3. If I do have lingering path problems, how can I eliminate those?

  4. Followed Mauthier’s suggestions
    4.1. took out rstan_options(autho_write = TRUE), still “cannot allocate vector…”
    4.2. removed stanmodel.rds
    RStudio error message as before during install: "Building R package from source requires installation of additional build tools. Do you want to install the additional tools now?”
    RStudio asked me this twice, I clicked yes, twice:
    Compilation argument:

C:/PROGRA~1/R/R-40~1.0/bin/x64/R CMD SHLIB file23ac3b7144af.cpp 2> file23ac3b7144af.cpp.err.txt 
Warning message:
In system(cmd) : 'make' not found
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
  cannot open file 'file23ac3b7144af.cpp.err.txt': No such file or directory```

Hi,
just to be sure, have you also done the new steps required with Rtools 4.0 to put Rtools on your path? It’s described in section " Putting Rtools on the PATH" there:
https://cran.r-project.org/bin/windows/Rtools/

I also had the error about ‘make’ not found because I first did not see that there was this extra step now.
Cheers

Could you try it outside of RStudio? Just open R (x64), and run an RStan example.

Is it loading a saved workspace from before? After opening R, run rm(list=ls()) and try again.

And what is ‘storage’?

mauthier, yes, I did, I followed the steps but the last step–installing from source–threw an error.
C:/Rtools/mingw_32/bin/gcc -I"C:/PROGRA~1/R/R-40~1.0/include" -DNDEBUG -Iyajl/api -D__USE_MINGW_ANSI_STDIO -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c base64.c -o base64.o
sh: C:/Rtools/mingw_32/bin/gcc: No such file or directory
make: *** [C:/PROGRA~1/R/R-40~1.0/etc/i386/Makeconf:222: base64.o] Error 127
ERROR: compilation failed for package ‘jsonlite’

  • removing ‘C:/Users/tschwoerer/Documents/R/win-library/4.0/jsonlite’
  • restoring previous ‘C:/Users/tschwoerer/Documents/R/win-library/4.0/jsonlite’
    Warning in install.packages :
    installation of package ‘jsonlite’ had non-zero exit status

Ran model from within R (x64) pointing to proper working directory and got the following:
Compilation argument:
C:/PROGRA~1/R/R-40~1.0/bin/x64/R CMD SHLIB file3f383f61ee.cpp 2> file3f383f61ee.cpp.err.txt
C:/Rtools/mingw_64/bin/g++ -m64 -std=c++1y -std=gnu++14 -I"C:/PROGRA~1/R/R-40~1.0/include" -DNDEBUG -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/Rcpp/include/" -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/RcppEigen/include/" -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/RcppEigen/include/unsupported" -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/BH/include" -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/StanHeaders/include/src/" -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/StanHeaders/include/" -I"C:/Users/tschwoerer/Documents/R/win-library/4.0/rstan/include" -DEIGEN_NO_DEBUG -D_REENTRANT -DBOOST_DISABLE_ASSERTS -DBOOST_PENDING_INTEGER_LOG2_HPP -include stan/math/prim/mat/fun/Eigen.hpp -std=c++1y -march=native -O3 -march=native -mtune=native -c file3f383f61ee.cpp -o file3f383f61ee.o
sh: C:/Rtools/mingw_64/bin/g++: No such file or directory
make: *** [C:/PROGRA~1/R/R-40~1.0/etc/x64/Makeconf:229: file3f383f61ee.o] Error 127
Error in file(con, “r”) : cannot open the connection
In addition: Warning message:
In file(con, “r”) :
cannot open file ‘file3f383f61ee.cpp.err.txt’: No such file or directory

Same compilation error after rm(list=ls())

What do you mean by "What is ‘storage’?

Seems like it is still looking for Rtools when it should be looking for rtools40 …

Are you absolutely, positively, 100% certain you have renamed or deleted your Makevars.win file, in Documents/.R/Makevars.win ? Because yes, it is looking for a compiler in C:/Rtools, which doesn’t exist.

And are you 100% certain you only have PATH="${RTOOLS40_HOME}\usr\bin;${PATH}" in Documents/.Renviron ?

And are you 100% certain you are starting with a clean workspace?

If so, then click start, and search for ‘environment variables’, then click ‘edit system environment variables’. Then click environment variables. Go to system variables, and edit Path. Remove only the line that has Rtools in it.

Then log out, log back in, and try Rstan again.

Could you also open R, and type Sys.getenv() and paste the output here?

  1. no, Makevars.win is in its location in C:\Users\tschwoerer\Documents.R but in Windows explorer the Type for that file is listed as Adobe CMS Extension
  2. yes
  3. yes
  4. Did not go this far as answer to 1. was “no”
    Then followed your next comment below

AGSDESKTOPJAVA C:\Program Files (x86)\ArcGIS\Desktop10.6
ALLUSERSPROFILE C:\ProgramData
APPDATA C:\Users\tschwoerer\AppData\Roaming
BINPREF C:/Rtools/mingw_(WIN)/bin/ ChocolateyInstall C:\ProgramData\chocolatey ChocolateyLastPathUpdate Wed Sep 26 17:18:32 2018 CommonProgramFiles C:\Program Files\Common Files CommonProgramFiles(x86) C:\Program Files (x86)\Common Files CommonProgramW6432 C:\Program Files\Common Files COMPUTERNAME ANC-MB0910A18 ComSpec C:\WINDOWS\system32\cmd.exe DriverData C:\Windows\System32\Drivers\DriverData GFORTRAN_STDERR_UNIT -1 GFORTRAN_STDOUT_UNIT -1 GOPATH C:\Users\tschwoerer\go GOROOT C:\Go\ HOME C:\Users\tschwoerer\Documents HOMEDRIVE P: HOMEPATH \ HOMESHARE \\anc-filesrv01.ua.ad.alaska.edu\employeeHome\30458916
LOCALAPPDATA C:\Users\tschwoerer\AppData\Local
LOGONSERVER \ANC-ADUA01
MSYS2_ENV_CONV_EXCL R_ARCH
NUMBER_OF_PROCESSORS 8
OneDrive C:\Users\tschwoerer\OneDrive
OS Windows_NT
PATH C:\rtools40\usr\bin;C:\Rtools\bin;C:\Program
Files (x86)\Common
Files\Oracle\Java\javapath;C:\Rtools\bin;c:\Rtools\bin;c:\Rtools\mingw_32\bin;C:\ProgramData\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\ProgramData\chocolatey\bin;C:\Go\bin;C:\Program
Files\Git\cmd;C:\Hugo\bin;C:\Users\tschwoerer\AppData\Local\GitHubDesktop\bin;C:\Users\tschwoerer\AppData\Local\Programs\Python\Python37;C:\Users\tschwoerer\AppData\Local\Programs\MiKTeX
2.9\miktex\bin\x64;%USERPROFILE%\AppData\Local\Microsoft\WindowsApps;
PATHEXT .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
PROCESSOR_ARCHITECTURE
AMD64
PROCESSOR_IDENTIFIER Intel64 Family 6 Model 58 Stepping 9,
GenuineIntel
PROCESSOR_LEVEL 6
PROCESSOR_REVISION 3a09
ProgramData C:\ProgramData
ProgramFiles C:\Program Files
ProgramFiles(x86) C:\Program Files (x86)
ProgramW6432 C:\Program Files
PSModulePath C:\Program
Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
PUBLIC C:\Users\Public
R_ARCH /x64
R_COMPILED_BY gcc 8.3.0
R_HOME C:/PROGRA~1/R/R-40~1.0
R_LIBS_USER C:\Users\tschwoerer\Documents/R/win-library/4.0
R_USER C:\Users\tschwoerer\Documents
RTOOLS40_HOME C:\rtools40
SESSIONNAME Console
SystemDrive C:
SystemRoot C:\WINDOWS
TEMP C:\Users\TSCHWO~1\AppData\Local\Temp
TMP C:\Users\TSCHWO~1\AppData\Local\Temp
UATDATA C:\WINDOWS\CCM\UATData\D9F8C395-CAB8-491d-B8AC-179A1FE1BE77
USERDNSDOMAIN UA.AD.ALASKA.EDU
USERDOMAIN UA
USERDOMAIN_ROAMINGPROFILE
UA
USERNAME tschwoerer
USERPROFILE C:\Users\tschwoerer
windir C:\WINDOWS

Also, here is my system variable window
image

Ok, again - You should not have anything defined in your Makevars.win right now. Move, delete, or rename it to something else. BINPREF is pointing to C:/Rtools, and it shouldn’t be.

Do you have a .Rprofile somewhere defining BINPREF?

It seems like there is some configuration, somewhere on your system (Makevars.win, .Renviron, .Rprofile) that is telling R that the compiler should be in BINPREF, defined somewhere as being C:/Rtools.

Edit: As for Windows saying that Makevars.win is an adobe CMS type, ignore it. Windows has a really rudimentary, stupid way of finding filetypes. It thinks Makevars.win is really a file named Makevars with extension win, which it apparently thinks is adobe. It is in reality a plain text file that defines compiler options on windows - hence why I said it should be removed from the equation for now.

.Rprofile contains:
Sys.setenv(BINPREF = "C:/Rtools/mingw_$(WIN)/bin/")

.Renviron contains:
PATH="${RTOOLS40_HOME}\usr\bin;${PATH}"

I can’t see what .Renviron contains, but I assume it is fine.

Remove Sys.setenv(BINPREF = “C:/Rtools/mingw_$(WIN)/bin/”) from your .Rprofile.

Also… move or rename Documents/.R/Makevars.win.

Ok see .Renviron above.

Model seems to compile properly now.

Is there anything else I should know to not run into the same issue again when updating to new versions or R? What about the edits to .Rprofile and the renaming of Makevars.win?

Do you happen to know which solution solved the problem, for future users to know?

As for what not to run: You do not need to specify a compiler. You may want to include a Makevars.win that has compiler optimizations defined, but not paths to compilers. There are some listed around here, and they look like CXX14FLAGS += -march=native -mtune=native -O3 -Wno-unused-variable -Wno-unused-function -fPIC if I recall correctly. That’s what I have on linux, unsure whether those options hold up on windows. Basically, just avoid modifying paths, or specifying which compiler to use, unless you know for certain what you’re doing. Future versions of R should continue working as far as I know.

Thanks a lot for sticking with me through this very long problem solving act.
Remove Sys.setenv(BINPREF = “C:/Rtools/mingw_$(WIN)/bin/”) from your .Rprofile.
And
moving or rename Documents/.R/Makevars.win.
solved the problem