Cmdstanr with Rtools 4.0 on Windows 10: mingw32-make.exe missing?

[Max_Mantei]

Hi!

I’ve got another cmdstanr question – although I think this might apply more generally to CmdStan as well.

I’m on Windows 10 and recently installed R 4.0 and the new Rtools 4.0 and uninstalled Rtools 3.5 prior to that. When I wanted to update CmdStan via cmdstanr::install_cmdstan(overwrite = TRUE) it said that it could not find mingw32-make. I couldn’t find anything like that in Rtools 4.0, so I installed Rtools 3.5 again and just added its \mingw_64\bin\ to my PATH. Now everything works, but it’s a bit weird to have Rtools 4.0 and 3.5 installed simultaneously.

There is probably something I’m missing about using CmdStan/cmdstanr with Rtools 4.0?

[ahartikainen]

Is the difference this

C:\RTools\RTools35\bin
C:\Rtools\RTools35\mingw_64\bin

C:\RTools\RTools40\usr\bin
C:\Rtools\RTools40\mingw64\bin

ahartikainen/stan_performance_testing/blob/master/.github/workflows/workflow-CmdStanPy.yml

[Max_Mantei]

Thanks for the quick answer.

Yes, I have these

C:\rtools40\usr\bin
C:\rtools40\mingw64\bin

in my PATH. But it can’t find mingw32-make.exe (checking mingw32-make --version, too) until I add

C:\rtools40\usr\bin
C:\rtools40\mingw64\bin
C:\rtools\mingw_64\bin

then it works fine…

Not sure where I messed up…

[rok_cesnovar]

Hm, its completely possible that Rtools 4.0 does not have mingw32-make. Will take a look and see if there is a workaround.

[Max_Mantei]

Thank you so much! It’s not urgent as the above approach seems to work for now.

I just thought that in future more and more (Windows) people with R 4.0 (and Rtools 4.0) installed will try cmdstanr (hopfully!) and might run into this problem as well.

[rok_cesnovar]

Your approach will work everytime, but its far from ideal.

@Max_Mantei

I just thought that in future more and more (Windows) people with R 4.0 (and Rtools 4.0) installed will try cmdstanr (hopfully!) and might run into this problem as well.

Definitely.

[rok_cesnovar]

Ok, found it. Its actually not that tricky.

You need to start the Rtools bash shell like the Rtools guide on installin package dependencies says https://github.com/r-windows/docs/blob/master/rtools40.md

image from the guide:

Then write

pacman -Syu mingw-w64-x86_64-make

and Enter, then you will be prompted to install. Then close the window.

Make sure your path has both

C:\rtools40\usr\bin
C:\rtools40\mingw64\bin

Open shell (or restart Rstudio) and you should be good to go.

cc-ing those that might get windows and cmdstan related questions: @wds15 @mitzimorris @ahartikainen

[Max_Mantei]

Ah! Cool, thanks! I tried pacman -Sy mingw32-make before, but I also had no idea what I was doing. I didn’t think I was that close… hahah

[wds15]

So does that mean that windows users have to follow these steps to get the needed make flavour?

Can you put this up on a wiki? Or could we even automate this for the users? It looks like it is easy in principle, but just another barrier which will stop a number of Windows users… sigh.

Would it be worthwhile to think about downloading binary TBB libraries (at least on windows)? I mean, we anyway already download the stanc3 binary.

(Maybe we move the discussion somewhere else)

Thanks for fiddling this out!

[rok_cesnovar]

@wds15

So does that mean that windows users have to follow these steps to get the needed make flavour?

Yes. Anyone using Rtools 4.0 will need to do that. Rstan will build the TBB lib its on way right?

@wds15

Can you put this up on a wiki?

I am slowly working towards a “How to install interface XYZ” page with clicking and all. I am slow with docs and writing (get easily distracted by coding :) ) but should be ready by the end of the week. I had to write some install instructions for a JMLR Stan GPU paper we submitted so I have some material to work with.

@wds15

Or could we even automate this for the users? It looks like it is easy in principle, but just another barrier which will stop a number of Windows users… sigh.
Would it be worthwhile to think about downloading binary TBB libraries (at least on windows)? I mean, we anyway already download the stanc3 binary.

You can actually do

pacman -S mingw-w64-x86_64-intel-tbb

I am not completly sure what that does or where it places the TBB lib. It is promising and will look into it. But that will only come in handy for 2.24 as 2.23 will fail to build if not using mingw32-make.

This way of installing things will be how R 4.0 will operate on Windows. Users will do the same thing for any lib they need (openssl, xml2 and the likes). So I think this is actually an improvement.

@wds15

(Maybe we move the discussion somewhere else)

I think this topic is just fine.

[wds15]

@rok_cesnovar

Rstan will build the TBB lib its on way right?

Not really. We rely on the RcppParallel package which brings the TBB luckily in binary form (though in a rather oldish version which does not seem to matter as things are greatly compatible).

@rok_cesnovar

This way of installing things will be how R 4.0 will operate on Windows. Users will do the same thing for any lib they need (openssl, xml2 and the likes). So I think this is actually an improvement.

If we can get this working smoothly, then this is indeed an improvement. I would lean towards grabbing the binary TBB instead of building it. Then users would not even need anymore the mingw32-make. However, if we want to keep compatibility with RTools < 4.0, then we should probably stick with the mingw32-make … or is this pacman thing also available on RTools 3.5?

[rok_cesnovar]

@wds15

or is this pacman thing also available on RTools 3.5?

No, this package manager (hence pac-man) is part of Msys2 which is what this Rtools 4.0 bash actually is.

This is the RTools workaround for a lack of a package manager on Windows. These are all the available packages: GitHub - r-windows/rtools-packages: Toolchains and libraries for R-4.0 to R-4.2

[rok_cesnovar]

RStudio project will most likely add Rtools bash in the terminal tab https://github.com/rstudio/rstudio/issues/6810

This will simplify installing by some margin.

[fweber144]

Just a minor update: According to the pacman docs,

pacman -Syu mingw-w64-x86_64-make

should be used instead of

pacman -Sy mingw-w64-x86_64-make

[rok_cesnovar]

Hi @fweber144,

you are correct, technically its better to add the u as that also updates the packages on your system to a newer version if it exists. Will update my comment.

Thanks!