RStan has been falling behind the current version of Stan. On CRAN now it is 2.21 and on github it is 2.26. Many seem to prefer RcmdStan, but I still prefer RStan of different reasons (such as a lot of code using Rstan, access to gradients, log_prob() and the Stan function block functions I want to unittest from R).
I was trying to find information to read up on the latest status on RStan and thoughts on its development but could not find anything except the discussion in October last year. Hence I open up this thread if also others might wondering the same. So two short questions:
Is there a status/plan for Rstan on CRAN with a version later than 2.21?
Is there a status/plan on Rstan with a stan version later than 2.26 on github/mc-stan?
On Github you can access the latest Rstan version and in most cases, you can install it without compiling. The package files and binaries are available under the rstan Github Actions for the experimental branch: R-CMD-check Ā· Workflow runs Ā· stan-dev/rstan Ā· GitHub
RStan is actually the most up-to-date interface to Stan. The experimental version is at the latest commit of the develop branch of Stan, and my fork tests all pending changes as well. However, releasing it to CRAN faces many challenges and the consequences of earlier decisions: mainly the backward incompatibility at the C++ level (C++ code generated by stanc2 should be compatible with the headers and C++ code from stanc3!) and the split of StanHeaders and Stan-to-C++ compiler (in rstan) into two separate packages that canāt be updated together per CRAN policies. RStan v2.26 will be a transition release that addresses those issues, with a lot of temporary patches for backward compatibility at least for CRAN reverse dependencies. This has been very challenging considering the large number of dependencies and requiring TBB in the newer versions of Stan that we need to push to all of them or as we did apply a workaround for successful build and tests of all dependencies.
StanHeaders v2.26 has been very close to CRAN, with only a couple packages failing a test check. It will be followed by rstan v2.26.
As @rok_cesnovar mentioned, we can host the source and binary packages of RStan v2.30 on our repo for testing after the release of v2.26.
Can you also provide instructions on how to install from the Actions?
Specifically, I would like to be able to install rstanarm with 2.30 and STANC+=--O1 and CXXFLAGS += -march=native -mtune=native enabled.
I assume I need to first install rstan with 2.30, and I can see something related to that in the Actions for experimental, but I donāt know how to install from actions. If I try to install with remotes::install_github("stan-dev/rstan",ref="experimental",subdir="rstan/rstan") it installs āStanHeadersā 2.21.0-7 from CRAN and then fails. If I first install remotes::install_github("stan-dev/rstan",ref="experimental",subdir="StanHeaders") and then try again installing RStan, I get g++: error: /home/ave/R/x86_64-pc-linux-gnu-library/4.2/rstan/lib//libStanServices.a: No such file or directory. Iām out of ideas what to try next.
Click on the one which matches your R version and OS (based on your output above would be R-release_ubuntu_20.04) and it will download a .zip file containing the binaries for rstan and StanHeaders.
To install them in R, you would first unzip the file and then run:
For adding the optimisation flags for rstanarm, you can add the CXXFLAGS += -march=native -mtune=native flags to your ~/.R/Makevars file
Unfortunately the process rstanarm uses for generating C++ from the Stan files doesnāt include functionality for passing flags to stanc, so thatās not possible just yet (unless you modify the package source)
Ah, then you donāt need to be installing from the Actions, since thatās only for installing binaries. The install from source code that you posted was correct
Looking at the stanc code in rstan experimental branch, it looks like --O is not supported. There is an explicit list of flags, but no flag for optimization.
@avehtari If youāre interested, I could create the binary packages for you, with custom optimizations. I use extreme optimizations on my server, including -DEIGEN_USE_BLAS + Intel MKL (and -DEIGEN_USE_MKL_ALL for some packages).
The source package is not associated with R version, unless we explicitly list a max R version in the DESCRIPTION which is not the case here. You may get this message if you install from a binary package that is built for a different version of R.
You can force any compiler flags you want in the ~/.R/Makevars or even add:
This was supported, by allow_optimizations argument, in an earlier version before this comment from @stevebronder.
Update: You can now force the experimental optimizations using the following option in R:
options(stanc.allow_optimizations = TRUE)
Note that āSome of these are not thoroughly tested and may not always improve a programās performance.ā (from stanc3 help)
This overrides the new rstan::stanc() argument (allow_optimizations = FALSE) and will force it on rstanarm if enabled before build/install (not tested yet)
It would be nice to support specifically the O1 optimizations. The experimental ones are pretty much known to make some bad choices (e.g. Iāve seen them move expensive function calls out of transformed data and into the model block)
@WardBrianrstanarm compilation fails if O1 optimizations are enabled.
Intel Compiler:
stan_files/polr.hpp(1757): error: no operator "[]" matches these operands
operand types are: stan::conditional_var_value_t<stan::scalar_type_t<Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>>, Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>> [ int ]
eta[(inline_pw_polr_n_sym149__ - 1)]));
^
Clang:
In file included from stan_files/polr.cc:3:
stan_files/polr.hpp:1757:35: error: type 'stan::conditional_var_value_t<local_scalar_t__, Eigen::Matrix<local_scalar_t__, -1, 1>>' (aka 'stan::math::var_value<Eigen::Matrix<double, -1, 1, 0>>') does not provide a
subscript operator
eta[(inline_pw_polr_n_sym149__ - 1)]));
^~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Iāme fine compiling myself, and I think itās better so that I can try different things, and also I do sometimes make changes to rstanarm code, so itās better if Iām able to compile from the source. I just need some instructions, as things are changing and the old instructions donāt work.
I see. I can parse or append stanc3 flags from STANCFLAGS environment variable, but this will be applied globally and may interfere or break R packages (e.g., if it includes an incompatible flag with the Stan code in the package - Also, automatic formatting returns Stan code, not C++). Though, itās fair to assume that only experienced users set STANCFLAGS.
The GitHub artifacts are compressed. You need to download and unzip the file.
If youād like to apply optimizations to all Stan code in R, add the following line in ~/.Rprofile:
options(stanc.allow_optimizations = TRUE)
If we support STANCFLAGS like CMDStan, you need to set it in ~/.Renviron. This is because packages might not inherit options or environment variables during installation. ~/.Rprofile/~/.Renviron are applied everywhere.
