RStan has been falling behind the current version of Stan. On CRAN now it is 2.21 and on github it is 2.26. Many seem to prefer RcmdStan, but I still prefer RStan of different reasons (such as a lot of code using Rstan, access to gradients, log_prob() and the Stan function block functions I want to unittest from R).
I was trying to find information to read up on the latest status on RStan and thoughts on its development but could not find anything except the discussion in October last year. Hence I open up this thread if also others might wondering the same. So two short questions:
Is there a status/plan for Rstan on CRAN with a version later than 2.21?
Is there a status/plan on Rstan with a stan version later than 2.26 on github/mc-stan?
On Github you can access the latest Rstan version and in most cases, you can install it without compiling. The package files and binaries are available under the rstan Github Actions for the experimental branch: Actions · stan-dev/rstan · GitHub
RStan is actually the most up-to-date interface to Stan. The experimental version is at the latest commit of the develop branch of Stan, and my fork tests all pending changes as well. However, releasing it to CRAN faces many challenges and the consequences of earlier decisions: mainly the backward incompatibility at the C++ level (C++ code generated by stanc2 should be compatible with the headers and C++ code from stanc3!) and the split of StanHeaders and Stan-to-C++ compiler (in rstan) into two separate packages that can’t be updated together per CRAN policies. RStan v2.26 will be a transition release that addresses those issues, with a lot of temporary patches for backward compatibility at least for CRAN reverse dependencies. This has been very challenging considering the large number of dependencies and requiring TBB in the newer versions of Stan that we need to push to all of them or as we did apply a workaround for successful build and tests of all dependencies.
StanHeaders v2.26 has been very close to CRAN, with only a couple packages failing a test check. It will be followed by rstan v2.26.
As @rok_cesnovar mentioned, we can host the source and binary packages of RStan v2.30 on our repo for testing after the release of v2.26.
Can you also provide instructions on how to install from the Actions?
Specifically, I would like to be able to install rstanarm with 2.30 and STANC+=--O1 and CXXFLAGS += -march=native -mtune=native enabled.
I assume I need to first install rstan with 2.30, and I can see something related to that in the Actions for experimental, but I don’t know how to install from actions. If I try to install with remotes::install_github("stan-dev/rstan",ref="experimental",subdir="rstan/rstan") it installs ‘StanHeaders’ 2.21.0-7 from CRAN and then fails. If I first install remotes::install_github("stan-dev/rstan",ref="experimental",subdir="StanHeaders") and then try again installing RStan, I get g++: error: /home/ave/R/x86_64-pc-linux-gnu-library/4.2/rstan/lib//libStanServices.a: No such file or directory. I’m out of ideas what to try next.
For adding the optimisation flags for rstanarm, you can add the CXXFLAGS += -march=native -mtune=native flags to your ~/.R/Makevars file
Unfortunately the process rstanarm uses for generating C++ from the Stan files doesn’t include functionality for passing flags to stanc, so that’s not possible just yet (unless you modify the package source)
@avehtari If you’re interested, I could create the binary packages for you, with custom optimizations. I use extreme optimizations on my server, including -DEIGEN_USE_BLAS + Intel MKL (and -DEIGEN_USE_MKL_ALL for some packages).
The source package is not associated with R version, unless we explicitly list a max R version in the DESCRIPTION which is not the case here. You may get this message if you install from a binary package that is built for a different version of R.
You can force any compiler flags you want in the ~/.R/Makevars or even add:
It would be nice to support specifically the O1 optimizations. The experimental ones are pretty much known to make some bad choices (e.g. I’ve seen them move expensive function calls out of transformed data and into the model block)
@WardBrianrstanarm compilation fails if O1 optimizations are enabled.
stan_files/polr.hpp(1757): error: no operator "" matches these operands
operand types are: stan::conditional_var_value_t<stan::scalar_type_t<Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>>, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>> [ int ]
eta[(inline_pw_polr_n_sym149__ - 1)]));
In file included from stan_files/polr.cc:3:
stan_files/polr.hpp:1757:35: error: type 'stan::conditional_var_value_t<local_scalar_t__, Eigen::Matrix<local_scalar_t__, -1, 1>>' (aka 'stan::math::var_value<Eigen::Matrix<double, -1, 1, 0>>') does not provide a
eta[(inline_pw_polr_n_sym149__ - 1)]));
I’me fine compiling myself, and I think it’s better so that I can try different things, and also I do sometimes make changes to rstanarm code, so it’s better if I’m able to compile from the source. I just need some instructions, as things are changing and the old instructions don’t work.
I see. I can parse or append stanc3 flags from STANCFLAGS environment variable, but this will be applied globally and may interfere or break R packages (e.g., if it includes an incompatible flag with the Stan code in the package - Also, automatic formatting returns Stan code, not C++). Though, it’s fair to assume that only experienced users set STANCFLAGS.
The GitHub artifacts are compressed. You need to download and unzip the file.
If you’d like to apply optimizations to all Stan code in R, add the following line in ~/.Rprofile:
options(stanc.allow_optimizations = TRUE)
If we support STANCFLAGS like CMDStan, you need to set it in ~/.Renviron. This is because packages might not inherit options or environment variables during installation. ~/.Rprofile/~/.Renviron are applied everywhere.