Rstantools: very slow compilation with R 4.0

I am currently building an R package with rstantools and I notice that the compilation of stan files became very slow since I moved to R4.0.

I built the R package with rstantools a year ago and I was able to install the package following the vignette within 5 minutes, either on Mac or Windows.

A year later, with now R 4.0.3 and RbuildTools 4.0, the compilation was still running after 2 hours
Today, I have just tried to build a simple lm package with the vignette and it was still running after 30 minutes while before it was compiling in a few seconds.

I really don’t know where the problem is. Any help is welcome !

N.B : Hopefully, my Mac is still under R 3.6.3 and I can compile there in a few minutes but I would like to solve this problem with R 4.0.

Thanks!

1 Like

I honestly have no idea, but hopefully @rok_cesnovar would have clue (and time to look into this).

A good thing to check would be if you are experiencing long build times also for non-stan packages (e.g. just trying to install some package with non-trivial C++ content from source).

This is definitely a sign of a misconfigured toolchain. Even models with 1000+ calls to lpdfs should compile in under a few minutes.

Does the model compile fast if you just compile it with rstan? Can you share an example of a model that compiles slow?

1 Like

Thank you @martinmodrak and @rok_cesnovar for your help! Indeed, the toolchain was not well configured. I followed the instructions from the rstan vignette and now the package compiles in around 10 min.

Even though it compiles in 10min, there is a long log during installation and maybe the compilation time can still be improved. If one of you has some time to have a quick look, the help is appreciated.

The link to the github R package: GitHub - kabarigou/StanMoMo: Bayesian Mortality Modeling with Stan
The aim of the package is to perform Bayesian mortality forecasting for actuaries, statisticians and demographers. The package is based on 5 stan files where each corresponds to one mortality model.

Any comment is appreciated !

2 Likes

5 models taking 10 minutes with rstan is somewhere in the ballpark of expected compile times.

Do you mean a long list of warnings? That is expected and those could be silenced but how to silence them depends on which of them pop up.

What you could do is prebuild the package for Windows/Mac using Github Actions so the installation would be really fast for your users. They would get the same experience as if the package was on CRAN.

1 Like

Thank you @rok_cesnovar for your answer.

Do you mean a long list of warnings? That is expected and those could be silenced but how to silence them depends on which of them pop up.

Indeed, a long list of not clear warnings but maybe these cannot be avoided. One can have a look with the command:

devtools::install_github('kabarigou/StanMoMo')

What you could do is prebuild the package for Windows/Mac using Github Actions so the installation would be really fast for your users. They would get the same experience as if the package was on CRAN.

That would be awesome ! How can I do this ? Should I simply run the command

R CMD INSTALL --build namepackage.tar.gz

on a Windows and Mac computer and then put these binaries in the repository ?

Thanks in advance for your help !

N.B : I think it would be useful to add a vignette somewhere explaining how to build Windows and Mac binaries to avoid compilation of users installing packages from source. This point was already asked in different posts (see here for instance) and this can be really helpful for users of Stan.

Yes, you need to build binary packages and place them somewhere your users can download them.
Maybe easier to do with pkgbuild like:

bin2 <- pkgbuild::build("./rstan/rstan", dest_path = tempdir(), vignettes = FALSE, manual = FALSE, binary = TRUE)

code from rstan/main.yml at 483781fedc83558eff2ab139bab4215883c9bf52 · stan-dev/rstan · GitHub

I would not put them in git as these binary packages that contain Stan files can be big, maybe rather as files on a release, kind of like we have tar.gz files here: Release v2.26.1 (17 February 2021) · stan-dev/cmdstan · GitHub (this is not a R package, just an example).

I would also suggest using Github Actions that give you free access to Windows/Mac machines (for open source). But if you have easy access to mac/win machines maybe not needed.

You can also use the drat package (CRAN - Package drat) to make your own “CRAN”, kind of like we do for GitHub - stan-dev/r-packages: Repository for distributing (some) stan-dev R packages

and then users can install using install.packages.

Good point, will make an issue on the r-packages repo.

1 Like

Thank you so much @rok_cesnovar for your detailed answer !

For the moment, I have Windows and Mac computers so I will just call the function pkgbbuild::build on both machines and create a new github repository where I store both binaries using the drat package, similar to the stan-dev/r-packages repository.

I would also suggest using Github Actions that give you free access to Windows/Mac machines (for open source). But if you have easy access to mac/win machines maybe not needed.

Can you give me some details how I can run R code on a Mac server using Github actions ? (or a link that shows the main procedure)

Thanks a lot!