BridgeStan provides a bridge from a Stan program to any language with a C foreign function interface (FFI), in order to expose log density evaluations and gradients of the contained Stan model.
So far BridgeStan has focused on a few core components, instead of features or doc.
Force a Stan model through a C interface to enable any higher level language with a C FFI to call BridgeStan. There are currently examples for Python and Julia.
Ensure calls from multi-threaded code (say from Julia) are handled correctly. Calls from parallelized code should also work. Much thanks to @wds15.
Tweak the Makefile to work on macOS, Linux, and Windows. Much thanks to @WardBrian.
Once Stan math issue #2743 is more widely available, BridgeStan should be entirely copy free. Right now the position vector is copied once for every call to the gradient function.
Thanks, @roualdes. I can attest to the utility of this package because it’s what Edward’s been using with Julia to evaluate Matt Hoffman’s new highly parallel samplers (CheeS, Snaper, MEADS). MEADS is particularly cool, being an adaptation- and tuning-parameter free version of Radford Neal’s new non-reversible acceptance sampler, which itself supports partial-momentum refresh Metropolis-adjusted Langevin (MALA).
This is very cool. One useful feature (I assume it’s a related problem, but perhaps not) might be access to definitions in the functions block. If I am doing straightforward but nonlinear curve-fitting, I often find myself defining the same function in Stan as well as in python, e.g., so I can plot smooth fits through the data.
RStan already lets you do this, so others have also asked for the functionality.
Yes, I experimented with getting it to work within pystan a while back, but I couldn’t get it working except for very simple cases. (And in any case I imagine that it wouldn’t really be possible with cmdstanpy – or even cmdstanr? – which now seem to be favoured.)
But are you implying that this in the roadmap for BridgeStan? Or something separate?
Thanks for that example. There is quite a bit of C/C++/Python interop so I suspect it’s possible but I imagine very different from R. Anyone out there with ideas or pointers?
I think if it can be made cffi-compatible a la BridgeStan then it is probably straightforward.
I think trying to expose the UDFs of a Stan program in something like bridgestan is not possible. The key idea that makes bridgestan work is that the model class always has some functions which are easy to expose to a C wrapper. The UDFs will be different for each and every model.
That said, in some sense the problem is actually easier, since you don’t need to worry about sharing memory with an instantiated object - all those functions can be called without instantiating a model. I suspect you could do it with something like the standalone functions trick @rok_cesnovar links to above and pybind11 documentation
Right. Plus there are issues of matching container types and memory layout from the arguments of the Stan function(s) to Python (or which ever higher level language). So exposing user defined functions is more complicated than what BridgeStan does.
@defjaf, I’m happy to help you work up a prototype if you want, though I myself would have to rely on help from either or both of Brian and/or Rok. If so, maybe a new thread would be best with a minimal example Stan program containing a user defined function of interest.
Bridgestan has had an overhaul internally and is now feature complete. An R interface was added, the remaining model class functionality was exposed, and important bugs around topics like exception handling have been fixed.
We still haven’t formally declared a versioned release, but it is moving toward being production ready. Documentation is one of the bigger remaining hurdles.
BridgeStan continues to move forward and is (hopefully) approaching battle tested.
Thanks to @Niko for finding a subtle bug, and big big up to @WardBrian and @stevebronder for squashing it within a day. They simultaneously solved another issue, found by @alphillips (I think), that showed up only on newer versions of gcc.
At the moment we’re not thinking of any release avenues besides Github. Using it from R/Python/Julia will always require downloading the C++ code from there, and a working copy of CmdStan (or possibly just Stan, we’re investigating loosening the requirements there)