Rstan on Windows

Interfaces RStan
1

As you all have noticed, there are a large number of people who are having problems with rstan 2.21.x on Windows. This thread is to try to bring those threads together.

Eliminate all -march=native clauses from CXX14FLAGS in the ~/.R/Makevars.win or ~/.R/Makevars files, which you can do by calling file.edit("~/.R/Makevars.win") and file.edit("~/.R/Makevars"). Even though the startup message says to consider adding it to Sys.setenv(LOCAL_CPPFLAGS), do not do that.

Note that the temporary URLs below will eventually die and I will need to regenerate / change them.

For R 3.x

If you are using R version 3.x, please do

install.packages(
"https://artifacts.r-hub.io/StanHeaders_2.21.0-6.tar.gz-6fc9f644607800c88f50865e72d8a81a/StanHeaders_2.21.0-6.zip",
repos = NULL, type = "win.binary")
install.packages(
"https://win-builder.r-project.org/EwLg0orTrr82/rstan_2.21.2.zip",
repos = NULL, type = "win.binary")

which should allow you to compile and run Stan models. However, I suspect that many of you will run into problems where the sampling fails, which we will need to follow up on.

We think you can get away with

CXX11FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2

but not -march=native.

For R 4.x

Please do

install.packages(
"https://artifacts.r-hub.io/StanHeaders_2.21.0-6.tar.gz-d74e0af7c9e06cba7d975cd8d3dbcf87/StanHeaders_2.21.0-6.zip",
repos = NULL, type = "win.binary")
install.packages(
"https://artifacts.r-hub.io/rstan_2.21.2.tar.gz-639222303bb62625e2349320b5adb986/rstan_2.21.2.zip",
repos = NULL, type = "win.binary")

which should allow you to compile and run Stan models. However, I suspect that many of you will run into problems where the sampling fails, which we will need to follow up on.

We think you can get away with

CXX14FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2

but not -march=native.

Stack overflow-like Problems

Your models, particularly if they are non-trivial, might crash or exhibit errors that say things like

==== C stack trace ===============================

At the moment, we are not sure if this problem still exists or how to solve it.

9 Likes
3

Fails on my machine (Windows 7, R 4.0.2).
I searched the output for errors and found:

collect2.exe: error: ld returned 1 exit status

This may be a long shot but I also posted Building from source on windows fails, cannot find -ltbb - ltbbmalloc · Issue #118 · RcppCore/RcppParallel · GitHub
which may be related because of dependencies???

4

Try with the updated OP that now has the binary install for R 4.x.

1 Like
5

Thanks very much for this!

It appears I can now run models again. I do get a weird error, but it does not appear to affect anything. The error is:

Warning message:
In file.remove(c(unprocessed, processed)) :
cannot remove file ‘C:\Users\d\AppData\Local\Temp\RtmpgRKi8f\file16a04c01cd4.stan’, reason ‘No such file or directory’

1 Like
6

Don’t worry about that.

7

Why would the binary work? What about winbuilder builds it such that rstan works?

I indeed was bitten by new bugs in my windows VM. But it was due to ‘spacing’:

E.g., I have C:/Users/Stephen Martin/foobar; and it fails at compile time because Martin/foobar is not a directory. That didn’t happen before.

Curious what winbuilder does differently to evade these problems?

1 Like
8

The StanHeaders binary gets rid of some stuff that we had before to workaround issues with the Eigen plugin but that created other issues when the path had spaces in it and that means that only a few packages have to work around the lack of the original work around. Anyway, those temporary binaries should compile things fine but we are still pretty much in the dark as to why sampling subsequently fails for some people some of the time who download R for Windows from CRAN where things are not failing.

9

I can confirm that the R 4.x binaries you posted above do the trick. However, I had to change my makevars.win from

CXX11FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-ignored-attributes -Wno-deprecated-declarations 
CXX14FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-ignored-attributes -Wno-deprecated-declarations

to

CXX14FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 
CXX11FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2

as suggested by @wds15 in some other thread. With the first makevars shown above, R(studio) crashed right after compilation when Stan tried to start sampling.

2 Likes
10

Removing -march=native makes it work on my computer too.

11

@bbbales2 not necessary, but does this happen in plain R as well? The -m* ones are telling the compiler it can use any of those instruction sets. Odd, that would imply -march=native makes mingw think it has access to instruction sets it shouldn’t actually have access to (which would be a bug in the compiler)

Would you and @paul.buerkner mind running the below and pasting the results in a gist with what CPU type you have? (Though I think this is mostly for my own curiosity)

system2(rstan:::get_CXX(), " -Q --help=target")
system2(rstan:::get_CXX(), "-mtune=native  -Q --help=target")
system2(rstan:::get_CXX(), "-march=native  -Q --help=target")
system2(rstan:::get_CXX(), "-march=native -mtune=native  -Q --help=target")

this stackoverflow posts says you can get the CPU type by going to the command prompt and running

wmic cpu get name

Side Note:
Dan Lemire has a blog about march vs mtune that is interesting

12

Intel® Core™ i5-6300U CPU @ 2.40GHz

none.txt (8.7 KB), No flags
mtune.txt (8.7 KB), -mtune=native
march.txt (8.7 KB), -march=native
both.txt (8.7 KB), -mtune=native -march=native
core2.txt (8.7 KB), -mtune=native -march=core2

Edit: The model worked with -mtune=native -march=core2 so I thought that might be the smallest diff to look through.

13

Could the URLs have expired so quickly? The installation of RStan itself goes just fine…

install.packages("https://win-builder.r-project.org/iT79L894j228/rstan_2.21.2.zip", repos = NULL, type = "win.binary")

But the installation of StanHeaders from its temporary URL…

install.packages("https://win-builder.r-project.org/521p7D65m24s/StanHeaders_2.21.0-6.zip", repos = NULL, type = "win.binary")

…yields the following:

trying URL 'https://win-builder.r-project.org/521p7D65m24s/StanHeaders_2.21.0-6.zip' Warning in install.packages : cannot open URL 'https://win-builder.r-project.org/521p7D65m24s/StanHeaders_2.21.0-6.zip': HTTP status was '404 Not Found' Error in download.file(p, destfile, method, mode = "wb", ...) : cannot open URL 'https://win-builder.r-project.org/521p7D65m24s/StanHeaders_2.21.0-6.zip'

Do we need a new url?

14

It does look like that url doesn’t work anymore. @bgoodri?

Also, @bgoodri instead of relying on temporary links I think you could download the binaries and tag a patch release on github and upload the binaries there. Even if you have to do that multiple times (if there are more fixes) that’s probably worth it instead of relying on temporary links. (Or you could upload them somewhere else as long as they don’t expire quickly.)

1 Like
15

Yes. Updated the OP with them.

16

The compiler flags I suggested are from my colleague who found out this problem with mingw:

So in short, AVX and AVX2 are broken under mingw which makes things unreliable when those instructions are used under Windows.

It sounds like we are finally resolving the Windows issues?

4 Likes
17

Thanks! That worked for me too.

1 Like
18

I’m a newbie here and a bit overwhelmed with the details, so please excuse the basic question:
Should I do file.edit("~/.R/Makevars.win") and file.edit("~/.R/Makevars") before or after installing the packages using the provided URLs (which will hopefully work until I try it out)?

Thanks!

19

@stevebronder what output do you want to see from those commands? They only return (invisible) 127.

20

It doesn’t matter. Just before trying to call stan.