Problems getting Stan to run after upgrading to R 4.0.2

2020-09-06T14:00:00Z
Hello
This seems to be a common problem but the threads are long and difficult to follow. Can’t get Stan to run now after multiple re-installs of R and R studio.
Tried fiddling with the makevars.win file; this is what I have in it now in the makevars file:

CXX14=$(BINPREF)g++ -O2 -march=native -mtune=native”

and I’ve tried

Sys.setenv(BINPREF = "C:/RBuildTools/4.0/mingw64/bin/g++.exe")

as my Rtools directory downloaded as RBuildTools. I keep getting this error message:

Error in compileCode(f, code, language = language, verbose = verbose) : 
  sh: C:/Rtools/mingw_64/bin/g++: No such file or directorymake: *** [C:/PROGRA~1/R/R-40~1.2/etc/x64/Makeconf:229: file123061c3410a.o] Error 127
In addition: Warning message:
In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  '-E' not found
Error in sink(type = "output") : invalid connection

Thanks for any advice.
regards
Chris

  • Windows 10
  • RStan 2.21.2
  • R 4.0.2
  • RStudio 1.3.1073

Hi Chris,

With RTools 4, you don’t want to be setting the CXX14 variable or the BINPREF variable, you also don’t want to use -march=native, since that can cause crashes.

The first step is to reset your Makevars to some ‘safe’ but optimised settings:

cat("CXX14FLAGS += -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2",file = "~/.R/Makevars.win", sep = "\n", append = FALSE)

Can you run that and then run the example model:

library(rstan)
example(stan_model,run.dontrun=TRUE,verbose=TRUE)

And post any line(s) that start with error:?

Also, small world! I finished my Biostats masters through Sydney uni last year

Thanks very much for your reply.
The error message I got now is

Error in compileCode(f, code, language = language, verbose = verbose) : 
  sh: C:/Rtools/mingw_64/bin/g++: No such file or directorymake: *** [C:/PROGRA~1/R/R-40~1.2/etc/x64/Makeconf:229: file1fc42e38522.o] Error 127
In addition: Warning messages:
1: In find.package(package, lib.loc, verbose = verbose) :
  package ‘base’ found more than once, using the first from
  “C:/PROGRA~1/R/R-40~1.2/library/base”,
  “C:/Program Files/R/R-4.0.2/library/base”
2: In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/RBUILD~1/4.0/usr/mingw_/bin/g++' not found

I finished the MBiostats last semester at Sydney Uni.
Are you Sydney based (if it’s not an inappropriate question on a public forum)?
Regards
Chris

No, I’m over in Perth. But always good to see more Australians on the forums!

This line:

sh: C:/Rtools/mingw_64/bin/g++

Indicates that R is looking for the old Rtools installation. What output do you get from the following:

Sys.getenv("PATH")
Sys.getenv("BINPREF")
readLines("~/.R/Makevars.win")
readLines("~/.Renviron")

Thanks
I get this with that request

 Sys.getenv("PATH")
[1] "C:\\RBuildTools\\4.0\\usr\\bin;C:\\Program Files\\R\\R-4.0.2\\bin\\x64;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath_target_289689671;C:\\Perl64\\site\\bin;C:\\Perl64\\bin;c:\\Rtools\\bin;c:\\Rtools\\mingw_32\\bin;C:\\Windows\\System32;C:\\Windows;C:\\Windows\\System32\\wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0;C:\\Windows\\System32\\OpenSSH;C:\\Program Files\\Intel\\WiFi\\bin;C:\\Program Files\\Common Files\\Intel\\WirelessCommon;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Users\\dalto\\AppData\\Local\\Microsoft\\WindowsApps"
> Sys.getenv("BINPREF")
[1] "C:/Rtools/mingw_$(WIN)/bin/"
> readLines("~/.R/Makevars.win")
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
  cannot open file 'C:/Users/dalto/Documents/.R/Makevars.win': No such file or directory
> readLines("~/.Renviron")
[1] "PATH=\"${RTOOLS40_HOME}\\usr\\bin;${PATH}\""

Something to do with RTools and RBuildTools?
Regards
Chris

It looks like the BINPREF variable is being set to the old RTools installation, which tells R where to look for the compilers. I think you probably have an .Rprofile file in your home directory that’s setting the BINPREF variable on startup.

If you don’t need any other settings in the file, you can just delete it via:

unlist("~/.Rprofile")

Otherwise just open the file in a text editor and remove the line mentioning BINPREF.

After deleting/editing the .Rprofile file, restart R and try your model again.

Additionally, you’re missing out on some compiler optimisations by not setting up a Makevars file. You can run the following to setup a Makevars:

cat("CXX14FLAGS += -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2",file = "~/.R/Makevars.win", sep = "\n", append = FALSE)

Now I seem to have some other problems!
1.RStudio keeps aborting the session when I try to open a file.
2.When using

cat("CXX14FLAGS += -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2",file = "~/.R/Makevars.win", sep = "\n", append = FALSE)

I get

Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file 'C:/Users/dalto/Documents/.R/Makevars.win': No such file or directory

My makevars file is here

"C:\Users\dalto\Documents\R\win-library\4.0\rstantools\include\sys\Makevars.win"

and contains

cat("CXX14FLAGS += -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2",file = "~/.R/Makevars.win", sep = "\n", append = FALSE)

3.When running a stan program I get

Warning message:
In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/RBUILD~1/4.0/usr/mingw_/bin/g++' not found

4.Running

library(rstan)
example(stan_model,run.dontrun=TRUE,verbose=TRUE)

I get

Warning messages:
1: In find.package(package, lib.loc, verbose = verbose) :
  package ‘base’ found more than once, using the first from
  “C:/PROGRA~1/R/R-40~1.2/library/base”,
  “C:/Program Files/R/R-4.0.2/library/base”
2: In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/RBUILD~1/4.0/usr/mingw_/bin/g++' not found

Sorry about this! Stan used to work well before I upgraded R and RStudio!
Regards
Chris

For the makevars file, you need to have it in your Documents folder, in a folder named ‘.R’

So you need to create a folder:

C:/Users/dalto/Documents/.R/

And then inside that folder create a file called Makevars.win which contains:

CXX14FLAGS += -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2

The cat(...) command was just to automatically create that file and it’s contents (not sure why it didn’t work), you don’t want to have the actual cat(...) command in there.

As for the models, those warnings are all safe to ignore (the g++ one will be fixed in the next release). Do the models all compile and sample without issue?

As another check, would you be able to post the output from:

devtools::session_info("rstan")

This is the output from

devtools::session_info("rstan")
Session info -----------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 4.0.2 (2020-06-22)
 os       Windows 10 x64              
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  English_Australia.1252      
 ctype    English_Australia.1252      
 tz       Australia/Sydney            
 date     2020-09-08                  

- Packages ---------------------------------------------------------------------------------------------------------------------
 ! package      * version   date       lib source                         
   assertthat     0.2.1     2019-03-21 [1] CRAN (R 4.0.2)                 
   backports      1.1.9     2020-08-24 [1] CRAN (R 4.0.2)                 
   BH             1.72.0-3  2020-01-08 [1] CRAN (R 4.0.0)                 
   callr          3.4.3     2020-03-28 [1] CRAN (R 4.0.2)                 
   checkmate      2.0.0     2020-02-06 [1] CRAN (R 4.0.2)                 
   cli            2.0.2     2020-02-28 [1] CRAN (R 4.0.2)                 
   colorspace     1.4-1     2019-03-18 [1] CRAN (R 4.0.2)                 
   crayon         1.3.4     2017-09-16 [1] CRAN (R 4.0.2)                 
   curl           4.3       2019-12-02 [1] CRAN (R 4.0.2)                 
   desc           1.2.0     2018-05-01 [1] CRAN (R 4.0.2)                 
   digest         0.6.25    2020-02-23 [1] CRAN (R 4.0.2)                 
   ellipsis       0.3.1     2020-05-15 [1] CRAN (R 4.0.2)                 
   evaluate       0.14      2019-05-28 [1] CRAN (R 4.0.2)                 
   fansi          0.4.1     2020-01-08 [1] CRAN (R 4.0.2)                 
   farver         2.0.3     2020-01-16 [1] CRAN (R 4.0.2)                 
   ggplot2      * 3.3.2     2020-06-19 [1] CRAN (R 4.0.2)                 
   glue           1.4.2     2020-08-27 [1] CRAN (R 4.0.2)                 
   gridExtra      2.3       2017-09-09 [1] CRAN (R 4.0.2)                 
   gtable         0.3.0     2019-03-25 [1] CRAN (R 4.0.2)                 
   inline         0.3.15    2020-09-07 [1] Github (bgoodri/inline@21105c7)
   isoband        0.2.2     2020-06-20 [1] CRAN (R 4.0.2)                 
   jsonlite       1.7.0     2020-06-25 [1] CRAN (R 4.0.2)                 
   labeling       0.3       2014-08-23 [1] CRAN (R 4.0.0)                 
   lattice        0.20-41   2020-04-02 [2] CRAN (R 4.0.2)                 
   lifecycle      0.2.0     2020-03-06 [1] CRAN (R 4.0.2)                 
   loo            2.3.1     2020-07-14 [1] CRAN (R 4.0.2)                 
   magrittr       1.5       2014-11-22 [1] CRAN (R 4.0.2)                 
   MASS           7.3-51.6  2020-04-26 [2] CRAN (R 4.0.2)                 
   Matrix         1.2-18    2019-11-27 [2] CRAN (R 4.0.2)                 
   matrixStats    0.56.0    2020-03-13 [1] CRAN (R 4.0.2)                 
   mgcv           1.8-31    2019-11-09 [2] CRAN (R 4.0.2)                 
   munsell        0.5.0     2018-06-12 [1] CRAN (R 4.0.2)                 
   nlme           3.1-148   2020-05-24 [2] CRAN (R 4.0.2)                 
   pillar         1.4.6     2020-07-10 [1] CRAN (R 4.0.2)                 
   pkgbuild       1.1.0     2020-07-13 [1] CRAN (R 4.0.2)                 
   pkgconfig      2.0.3     2019-09-22 [1] CRAN (R 4.0.2)                 
   pkgload        1.1.0     2020-05-29 [1] CRAN (R 4.0.2)                 
   praise         1.0.0     2015-08-11 [1] CRAN (R 4.0.2)                 
   prettyunits    1.1.1     2020-01-24 [1] CRAN (R 4.0.2)                 
   processx       3.4.4     2020-09-03 [1] CRAN (R 4.0.2)                 
   ps             1.3.4     2020-08-11 [1] CRAN (R 4.0.2)                 
   R6             2.4.1     2019-11-12 [1] CRAN (R 4.0.2)                 
   RColorBrewer   1.1-2     2014-12-07 [1] CRAN (R 4.0.0)                 
   Rcpp           1.0.5     2020-07-06 [1] CRAN (R 4.0.2)                 
   RcppEigen      0.3.3.7.0 2019-11-16 [1] CRAN (R 4.0.2)                 
 D RcppParallel   5.0.2     2020-06-24 [1] CRAN (R 4.0.2)                 
   rlang          0.4.7     2020-07-09 [1] CRAN (R 4.0.2)                 
   rprojroot      1.3-2     2018-01-03 [1] CRAN (R 4.0.2)                 
   rstan        * 2.21.2    2020-07-27 [1] CRAN (R 4.0.2)                 
   rstudioapi     0.11      2020-02-07 [1] CRAN (R 4.0.2)                 
   scales         1.1.1     2020-05-11 [1] CRAN (R 4.0.2)                 
   StanHeaders  * 2.21.0-6  2020-08-16 [1] CRAN (R 4.0.2)                 
   testthat       2.3.2     2020-03-02 [1] CRAN (R 4.0.2)                 
   tibble         3.0.3     2020-07-10 [1] CRAN (R 4.0.2)                 
   utf8           1.1.4     2018-05-24 [1] CRAN (R 4.0.2)                 
   V8             3.2.0     2020-06-19 [1] CRAN (R 4.0.2)                 
   vctrs          0.3.4     2020-08-29 [1] CRAN (R 4.0.2)                 
   viridisLite    0.3.0     2018-02-01 [1] CRAN (R 4.0.2)                 
   withr          2.2.0     2020-04-20 [1] CRAN (R 4.0.2)                 

[1] C:/Users/dalto/Documents/R/win-library/4.0
[2] C:/Program Files/R/R-4.0.2/library

 D -- DLL MD5 mismatch, broken installation.

and running a stan model I still get

Warning messages:
1: In find.package(package, lib.loc, verbose = verbose) :
  package ‘base’ found more than once, using the first from
  “C:/PROGRA~1/R/R-40~1.2/library/base”,
  “C:/Program Files/R/R-4.0.2/library/base”
2: In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/RBUILD~1/4.0/usr/mingw_/bin/g++' not found

Thanks again for your help!

But the models do compile and sample now with only this message!

Warning message:
In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/RBUILD~1/4.0/usr/mingw_/bin/g++' not found

Thanks!

Great, all of those warnings are safe to ignore, so you’re good to go

1 Like

For posterity, I thought I’d mention within this thread that I had every issue exactly as listed and tested by Chris above. Unfortunately, after following all of the suggestions, I still couldn’t get the example model to compile.

However, after going through these instructions and then also adding to my makevars file the line:

CXX14 = C:/Rtools40/mingw64/bin/g++

…everything worked out.

My makevars.win file now reads:

CXX14FLAGS += -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2
CXX14 = C:/Rtools40/mingw64/bin/g++

I took this advice from an older comment by @bgoodri here: G++ Compile Error With R Package

Hi Roy,

Just to check some reasons for why it wasn’t working initially (in case it’s a new issue I need to be aware of), can you post the outputs from:

Sys.getenv("BINPREF")
Sys.getenv("PATH")
Sys.which("make")
readLines("~/.Renviron")
readLines("~/.Rprofile")

Hi Andrew,

Thanks for the follow up. I admit I have little idea what is going on here and I was a little afraid I might unintentionally lead someone astray. Here is what I get running those commands:

> Sys.getenv("BINPREF")
[1] "C:/Rtools/mingw_$(WIN)/bin/"

> Sys.getenv("PATH")
[1] "C:\\rtools40\\usr\\bin;C:\\Program Files\\R\\R-4.0.2\\bin\\x64;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath_target_325312;C:\\Windows\\System32;C:\\Windows;C:\\Windows\\System32\\wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0;C:\\Program Files (x86)\\Windows Live\\Shared;C:\\Program Files\\Git\\cmd;C:\\Windows\\System32\\OpenSSH;C:\\Program Files\\McAfee\\Solidcore\\Tools\\GatherInfo;C:\\Program Files\\McAfee\\Solidcore\\Tools\\ScAnalyzer;C:\\Program Files\\McAfee\\Solidcore;C:\\Program Files\\McAfee\\Solidcore\\Tools\\ScGetCerts;C:\\Users\\rmartin\\AppData\\Local\\Microsoft\\WindowsApps"

> Sys.which("make")
                              make 
"C:\\rtools40\\usr\\bin\\make.exe" 

> readLines("~/.Renviron")
[1] "PATH=\"${RTOOLS40_HOME}\\usr\\bin;${PATH}\""

> readLines("~/.Rprofile")
[1] "Sys.setenv(BINPREF = \"C:/Rtools/mingw_$(WIN)/bin/\")"

Also, not sure if relevant, but I run Stan on an intel Xeon workstation (2 processors), which is maybe different from the average user. In the past, I think I’ve needed to fiddle a bit with the makevars settings relative to the usual recommendations for those settings, but unfortunately I forget the details. For example, I’m not sure if that was because of the Xeon setup, or just the general ‘march=native’ issues.

Finally, in case it helps, here is the output of errors I get when I run

library(rstan)
example(stan_model,run.dontrun=TRUE,verbose=TRUE)

without adding the

CXX14 = C:/Rtools40/mingw64/bin/g++

to the makevars.win file.

Compilation ERROR, function(s)/method(s) not created!
Error in compileCode(f, code, language = language, verbose = verbose) : 
  sh: C:/Rtools/mingw_64/bin/g++: No such file or directorymake: *** [C:/PROGRA~1/R/R-40~1.2/etc/x64/Makeconf:229: file9ce4183c6852.o] Error 127
In addition: Warning messages:
1: In find.package(package, lib.loc, verbose = verbose) :
  package ‘base’ found more than once, using the first from
  “C:/PROGRA~1/R/R-40~1.2/library/base”,
  “C:/Program Files/R/R-4.0.2/library/base”
2: In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/rtools40/usr/mingw_/bin/g++' not found

Thanks again,

Roy

Hi Roy,

That’s an easy fix thankfully! You were getting those errors because you’ve set the BINPREF variable in your .Rprofile file.

If you follow the instructions I posted above to delete the file, you won’t need to set the CXX14 variable.

Cheers
Andrew

1 Like

Ohhhh OK! I actually thought I did the delete the .Rprofile by running unlist("~/.Rprofile"), but I’ll fiddle around and try it again.Great to know what the underlying issue was, in any case!

-Roy

I had the same issue from above, and after switching withr to 2.2.0 and running:

example(stan_model,run.dontrun=TRUE,verbose=TRUE)

Warning messages:
1: In find.package(package, lib.loc, verbose = verbose) :
package ‘base’ found more than once, using the first from
“C:/Users/NMURAV~1/DOCUME~1/R/R-40~1.2/library/base”,
“C:/Users/nmuravskaya/Documents/R/R-4.0.2/library/base”
2: In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
‘C:/RBUILD~1/4.0/usr/mingw_/bin/g++’ not found

Hi Namurava,

None of those are errors that imply that the fitting failed. Were there any other errors or did the model finish running?