I develop R packages on the side and I’ve been trying to understand the differences between RStan, cmdstanr/cmdstan and bridgestan. I’m trying to figure out what use-cases each package is best suited for and would appreciate any insights.
I did look through the JOSS paper and this post BridgeStan 1.0.0 released , but it seems that things have changed since those links were posted and I’m unsure of what the current status is.
I understand that using cmdstan(r) that you can use the latest version of Stan and in my personal testing, the latest version of cmdstanr + stan is significantly faster and more storage friendly than the latest version of RStan.
Furthermore, you can now access functions implemented in Stan in R via cmdstanr. This implies that you might be able to bypass RCCP and do everything in Stan, even if you’re programming a non-Bayesian modelling package.
Here are some concrete questions:
If I want to implement a performant non-parametric modelling package in R, should I use briddgestan, cmdstanr or RCCP? Would I still be able to run tests on my code to ensure its correctness, the way I can for RCCP?
Would it be appropriate to say that bridgestan is a good fit if I want my package to be usable across python, R and more?
If I were to use Stan to program the demanding portions of my package, should I minimize the number of times that data is passed between R and the Stan program? Or does this not matter? For example, if I had to optimize a function for 10 iterations and evaluate the change in the loss function at each iteration, should everything be ideally done in Stan or would there be no appreciable difference with optimizing in Stan then evaluating the change in the loss function in R?
Some of this is easier to understand historically:
First, there was RStan. RStan works by directly interfacing the C++ that Stan is built on top of with R. This lets it do sampling etc as you’d expect, but also some powerful things like call user-defined Stan functions in a model or evaluate the log probability and gradient of the model. Unfortunately, this power combined with CRAN policies make it very difficult to keep up to date with newer releases of Stan.
CmdStanR is in theory much simpler, because rather than hooking up to the C++ it is just running a program and opening the output files. It is not on CRAN (yet?), which also helps it stay much more up-to-date. But, because it wasn’t directly interfacing with the C++, it lacked (the past tense will be explained later) those extra features beyond just running Stan’s algorithms.
But those features, particularly evaluating the log density, are very useful for algorithmic research and using Stan models with algorithms that aren’t (yet?) in Stan. That’s why bridgestan was built, with a slightly simpler architecture than RStan that makes it easier to keep updated and also provides a unified API in other languages like Python and Julia. It doesn’t allow you run the algorithms like sampling, or to call user-defined functions, just the log density (and its gradient, hessian, etc). It is also not on CRAN.
Since bridgestan was built, CmdStanR has gained the optional ability to interface directly with the C++ of the Stan model to gain those extra features that RStan has.
So, if you’re just using R, and don’t care about your package being on CRAN, CmdStanR is a one stop shop at this point. As you note, bridgestan would let you design against a similar API in other languages. And for CRAN uploads, RStan is the only choice still. For your third question, it really will depend on the size of the data
Also, no plans to put it on CRAN, despite basically everyone asking us to.
Pro: We can release whenever we want. CRAN is very restrictive because of (a) package size, (b) lack of dependency management, and (c) ability of third-party packages to write arbitrary unit tests against your output.
Con: Some people work for organizations that only let their employees run CRAN packages. They’re out of luck. Having said that, I don’t know what company would allow someone to use a CRAN package where the first step in using it is downloading a third-party application either as a binary or building it from scratch.
BridgeStan is using lower-level foreign function interfaces than either RStan or PyStan. This makes it much easier to install and much more compatible with changing R versions and platforms. It also makes it very easy to share memory.
I see… Thank you for the informative response and background history.
Out of curiosity, would it be difficult to develop an R package that’s compatible with all 3 of RStan, cmdstanr and bridgestan; or would I be pushed to choose between RStan/cmdstanr vs bridgestan? Perhaps I’m looking at this the wrong way, but from a developer point of view–the idea of programming the core of the modelling software once and only having to make small adjustments to interface with different programming languages sounds quite attractive.
I know that brms is compatible with both RStan and cmdstanr and this compatibility has been advantageous to me as a user…
Do you have an example of when cmdstanr/RStan might handle data transfers better than bridgestan? When bridgestan is better?
If all you need is the log density, you could write a package which is compatible with all three. If you need access to things like HMC sampling, bridgestan isn’t going to give you that.
I’m not sure we’ve done super rigorous comparisons of bridgestan and what RStan/CmdStanR provide for log density evaluation. My expectation is that bridgestan will be slightly easier to use/install, since it doesn’t need RCPP at all, and we’ve tried to make it as fast as possible. If you need second order autodiff, I believe cmdstanr always uses finite differences, but bridgestan lets you choose between finite diff and nested autodiff.
What do you need from Stan? bridgestan just gives you log density and gradient and Hessian evaluation, plus transforms. RStan and cmdstanr both are set up to do inference. Only bridgestan can run in Python, though cmdstanpy is the reference implementation on which cmdstanr was based.
I would suggest minimizing dependencies in code as it’ll lead to less documentation, less testing, and less maintenance.
I think part of the trouble for me–and the reason why it’s difficult to respond to your question–is that I’m mixing between: (1) what would be the best tool (RStan/cmdstanr vs. RCPP vs. bridgestan) to get the job done; (2) how I can get more comfortable with Stan; (3) how do I future-proof whatever I do.
At the moment, it seems to me that doing primarily RStan/cmdstanr–and then RCPP, if absolutely necessary–would be the best fit for my purposes, assuming that all the different linear algebra operations and etc I need are already supported.
I’m thinking this because although trying to make Stan do everything for me is introducing complexity, if I were to extend my project with a Bayesian component, then having Stan already in-place should probably make things easier. (I’m also working in a space where there isn’t necessarily a likelihood, but there’s definitely a lot of linear algebra going around.)