R 4.0, rstan, and you

[Stephen_Martin]

R 4.0 has released, and you are wanting to use rstan. Your models no longer run, you are getting cryptic error messages, and want to post here about it.

With R4.0, and the latest compiler configuration, you must configure rstan differently. Configurations from previous versions are breaking under R 4.0. Try these things first, as they generally solve any problems I have seen others report, and problems I have observed under OSX Mojave, Catalina, and Windows 10.

Summary

R 4.0 does not require manual configuration of compiler paths. However, previous R versions may have, and those configurations are breaking rstan on R 4.0. These instructions simply remove those configurations. Most problems I have seen so far about rstan on R 4.0, from both Windows and osx users, stem from these configuration files. Files affected: Makevars, Makevars.win, .Rprofile, .Renviron . These should not be setting compiler, include, or other paths unless you know what you are doing. Compiler optimization options are fine, and options that do not affect compiler paths are fine.

This is intended for those who used rstan before 4.0, and updated to 4.0

Windows 10

• Uninstall any previous Rtools versions (e.g., Rtools35).
• Move, delete, or rename:
  • Documents/.Renviron
  • Documents/.R/Makevars.win
  • Documents/.Rprofile
  • This will remove any previous configurations from the equation altogether.
• Install Rtools 40. Do not modify the default installation options.
• If you have not already, run this in R:
  writeLines('PATH="${RTOOLS40_HOME}\\usr\\bin;${PATH}"', con = "~/.Renviron")

You should now have only a Documents/.Renviron file.

Try compiling and running a Stan model. At this point, it should compile.

Rstan for Windows on R 4.0, with Rtools40, does not require Makevars.win to function. It does not require any compiler path, bin path (e.g., BINPREF), or include path defined.

If there are any compiler optimizations you wish to use, you can safely add it to Makevars.win. If you had custom functions or options in .Rprofile, you may return these to the file. Just ensure you are not modifying any paths related to compilation.

You may wish to add the following line to Documents/.R/Makevars.win for optimization:
CXX14FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function

OSX (Mojave, Catalina)

• Uninstall r-macos-rtools ; it is now unneeded.
• Undo the configurations made by macos-rtools:
  • Open spotlight, and launch terminal.
  • cd ~
  • mv .Renviron .Renviron.bak (this renames .Renviron, so R does not use it)
  • cd .R
  • mv Makevars Makevars.bak (this renames Makevars)
• You should have xtools installed already. If not, open terminal and run xcode-select --install and follow the prompts.

Open R, and compile/run a Stan model. It should now compile fine.

Like with Windows, rstan on R 4.0 does not require Makevars at all (at least, in all my testing).

At this point, you may want to add in compiler optimizations, such as CXX14FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function to your ~/.R/Makevars file.

Any other customizations you had in Renviron or Makevars can be returned to them as long as they do not specify directories. You do not need to specify includes, library, or compiler paths on R 4.0 with Xcode in order for rstan to function.

Edit: If you are on OSX and using Rstudio, you may have a crash when using multiple cores

This is not an rstan problem, but an Rstudio problem. See the fix here:

R 4.0.0 and CRAN MacOS binaries

I also was having issues with multiple chains running in parallel in MacOS using latest R-4.0.0 and RStan. I’ve found out that this is a problem of the package parallel from R-4.0.0 I’ve found this recently issue in RStudio’s GitHub that solved my problem: cl <- parallel::makeCluster(2, setup_strategy = "sequential") If this works, you could add the following to your ~/.Rprofile until this is fixed in R/RStudio: ## WORKAROUND: https://github.com/rstudio/rstudio/issues/6692 ## Revert to 'se…

[andrjohns]

Thanks Stephen, this is great! I’ve pinned it in the RStan category

[LailaFr]

Thank you very much, this solved my problems (R.4, Catalina)!

[mbschrimpf]

Hi,
I’m trying to install and run RStan for the first time, and I’m having a hard time figuring out how many of the RStan Getting Started instructions have changed with the release of R 4.0. I tried doing everything on that page, then tried reinstalling Rtools40 and following the instructions here. In both cases I get the following error message when I try running the eight schools example:

Error in file(con, “r”) : cannot open the connection
In addition: Warning messages:
1: In system(cmd, intern = !verbose) :
running command ‘C:/PROGRA~1/R/R-40~1.0/bin/x64/R CMD SHLIB file32e83919685e.cpp 2> file32e83919685e.cpp.err.txt’ had status 1
2: In file(con, “r”) :
cannot open file ‘file32e83919685e.cpp.err.txt’: No such file or directory
Error in sink(type = “output”) : invalid connection

There are various posts referring to older compiling errors, but I can’t seem to find anyone else who has posted about errors related to these specific files (and I don’t know what those files are supposed to be for, anyway). I’m also not sure if the lack of those files are my main issue.

I’m on Windows 10, 64-bit, using RStudio v1.2.5042.

Thanks for any help you can provide!
Michael

[andrjohns]

Hi Michael,

Can you run:
example(stan_model, run.dontrun = TRUE, verbose=TRUE)

And post the line that starts with error: ?

[mbschrimpf]

Hi, thanks for responding!

When I run that line it returns:

Error in file(con, "r") : cannot open the connection
In addition: Warning messages:
1: In find.package(package, lib.loc, verbose = verbose) :
package ‘base’ found more than once, using the first from
“C:/PROGRA~1/R/R-40~1.0/library/base”,
“C:/Program Files/R/R-4.0.0/library/base”
2: In file(con, "r") :
cannot open file 'fileaa027c521b7.cpp.err.txt': No such file or directory

I used to have an older installation of R (3.6 something), but I deleted that while trying to get Stan to work yesterday. I’m not sure where the first path is pointing - I can only find one R installation, and it’s in the second path.

-Michael

[andrjohns]

Further up there should be a line that starts with:

g++.exe: error: 

Can you post that one as well?

[mbschrimpf]

Ah, whoops - missed that one (different color):
g++.exe: error: Schrimpf/Documents/R/win-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp: No such file or directory make: *** [C:/PROGRA~1/R/R-40~1.0/etc/x64/Makeconf:229: fileaa027c521b7.o] Error 1

-Michael

[andrjohns]

Yep, that’s an issue we have at the moment when using RStan 2.19 with StanHeaders 2.21.

One workaround is to not load the RStan library, and just use the rstan:: qualifier instead.

So rather than:

library(rstan)
stan(model, data)

Use:

rstan::stan(model, data)

[mbschrimpf]

Now I’m thinking that maybe this has to do with the fact that Windows forces me to use my first [space] last name for my user directory? I’ve tried editing that, but the most recent updates to Windows 10 make that impossible, as far as I can tell.

[mbschrimpf]

Perfect!

@andrjohns

So rather than:

library(rstan)
stan(model, data)

Use:

rstan::stan(model, data)

That worked.

Thanks again for your help!

[Faith]

@andrjohns

example(stan_model, run.dontrun = TRUE, verbose=TRUE)

I just had the same problem as @mbschrimpf, and your solution @andrjohns of using rstan::stan() worked for me too. Thanks a lot!

[jonah]

@Faith Glad that works! We’ll be releasing a new version of rstan sometime this month that will fix this, so using the rstan:: prefix shouldn’t be necessary for much longer.

[stat-cz]

Hi Andrjohns
I try to run example(…). Then i also get the same error. And it still didn’t work when i run “rstan::stan(model,data)”.

make: *** [D:/ProgramFiles/...../Makeconf:229: file325024c96541.o] Error 127
Error in file(con, "r") : cannot open the connection
In addition: Warning messages:
1: In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\...RtmpwnxDM2\file3250af149e2.stan', reason 'No such file or directory'
2: In file(con, "r") :
  cannot open file 'file325024c96541.cpp.err.txt': No such file or directory

[ADC]

@andrjohns - Hi! I am trying to install rstan for the first time in RStudio 4, and am struggling to implement instructions online because I am not sure what is not updated. Can you please help?
RStudio is returning error messages when I install numerous packages “there is no package called x”

[spinkney]

I just went through installing everything on a cloud server. YMMV, but I had to add
install.packages(c("remotes", "knitr", "xml2", "devtools") before installing the dev version (if that’s what you want)

remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan", build_opts = "")

[andrjohns]

I was planning on updating the install instructions over the weekend, but no better time for a first draft!

I’m going to assume that you’re on windows, let me know if that’s not the case.

Install RStan on R4.0 (Windows)

Remove old RStan configuration

If you had previously installed RStan under R3.6, you may have some configuration files left over that will cause issues under R4.0.

The first thing to check is whether the BINPREF environment variable is pointing to the old RTools installation by running Sys.getenv("BINPREF").

You can skip to the next section if your output looks like the following:

Sys.getenv("BINPREF")
[1] ""

However, if your output looks like:

Sys.getenv("BINPREF")
[1] "C:/Rtools/mingw_$(WIN)/bin/"

Then you need to check whether your .Rprofile file has been configured to set this variable on R startup. If you run the following command and get the output:

readLines("~/.Rprofile")
[1] "Sys.setenv(BINPREF = "C:/Rtools/mingw_$(WIN)/bin/")"

Then you need to delete your .Rprofile file. You can get the location of this file by running:

file.path(Sys.getenv("HOME"), ".Rprofile")

Delete that file and then restart R. If BINPREF is no longer set, then you should get the following output:

Sys.getenv("BINPREF")
[1] ""

RTools4 Installation

The next step is to install RTools4 following the instructions here: https://cran.r-project.org/bin/windows/Rtools/

Make sure that you follow the instructions that put RTools on the path. Repeated here for clarity:

writeLines('PATH="${RTOOLS40_HOME}\\usr\\bin;${PATH}"', con = "~/.Renviron")

Then restart R and verify that RTools is being found via:

Sys.which("make")
## "C:\\rtools40\\usr\\bin\\make.exe"

And that you can install packages from source:

install.packages("jsonlite", type = "source")

Install RStan

Install Packages

You can now install RStan via:

install.packages("rstan", repos = "https://cloud.r-project.org/",
                 dependencies = TRUE)

Before moving to the next section verify that the following packages are at or above the indicated version:

• rstan: 2.21.2
• StanHeaders: 2.21.0-6
• inline: 0.3.16

Optimisations

You can speed up the estimation of your models by enabling some compiler options. To do this, you should add the following to your Makevars.win file:

CXX14FLAGS += -mtune=native -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2

Note that earlier installation guides also recommended the -march=native flag, but this will cause R to crash with RTools4 and is no longer recommended.

If you have other CXXFLAGS, CXX11FLAGS, or CXX14FLAGS lines already in your Makevars.win file, make sure that you delete those first. If you do not already have a Makevars.win file, or do not know where it is, then you can create this via R:

dotR <- file.path(Sys.getenv("HOME"), ".R")
if (!file.exists(dotR)) dir.create(dotR)
M <- file.path(dotR, "Makevars.win")
if (!file.exists(M)) file.create(M)
cat("\nCXX14FLAGS += -O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2",
     file = M, sep = "\n", append = FALSE)

Verify Installation

To verify your installation, you can run the RStan example/test model:

library(rstan)
example(stan_model,run.dontrun = TRUE)

The model should then compile and sample. You may also see the warning:

Warning message:
In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  'C:/rtools40/usr/mingw_/bin/g++' not found

This is safe to ignore and will be removed in the next RStan release.

Encountering Errors

If you are unable to install RStan or successfully run the example/test model, then please open a new thread on the Stan forums (https://discourse.mc-stan.org) with a description of your problem and any error message, and we will help troubleshoot your installation.

[andrjohns]

@jonah @bgoodri

I’ll be working on updating the installation instructions on the wiki over the weekend. The above is a draft of the instructions for working with R4.0 on Windows and identifying some of the common issues, can you have a look and see if there’s anything that I’ve missed?

Also, are there any specific things that I should put in the Linux & Mac writeups? The following is all that I’m aware of:

Linux:

• c++14 compatible compiler

Mac

• Not recommended with R3.6
• Use the r-macos-rtools package
• No -march=native flag

@torkar Was there anything else you’ve come across on Mac that should go in the install guide?