I have a function that specifies a stan model. I want to be able to include custom input at a few specific locations. Are there any approaches that would let me pre-compile the rest of the model? If there was a way to access elements outside a function, without explicitly passing them, this would probably work…
The scoping rules prevent accessing anything outside of user-defined functions except other user-defined functions. I suspect the answer is that it is not possible, but what do you mean by including custom input?
At present, it’s just inserting specified raw stan code at a couple of points. Any rough thoughts on how much overhead is incurred if I pass a few unnecessary matrices to a function, and that function is used many times per iteration step?
I would say there is not a noticeable amount of overhead because matrices are passed by reference to user-defined functions. You can use the #include filename mechanism to inject another file that contains Stan code, but it is very flaky. The # sign has to be flush left, there probably cannot be any spaces in filename, and there cannot be anything (not even whitespace) to the right of filename on that line.
1 Like
There’s no way to compile bits of Stan programs and link them. At least through our interfaces.
What you can do is what @bgoodri did for RStanArm, which is have conditional parts of models that are controlled with flags passed in as data. Then you can compile once and access different parts of the model.
Thanks Bob, yes I already have plenty of conditionals via data, but I want to allow basically anything that can be done in Stan as the time step in a stochastic differential. I had the idea that I might just be able to compile this separately as a function at run time, and reference that function from the pre-compiled model – am I misguided in this thinking?
In theory that might be possible, but we don’t have anything to support those linked builds in the interfaces. You can compile with a flag that lets you define functions externally, but I believe that’s only being done with statically linked external functions.
@seantalts or @syclik might know more about how to do this.
I haven’t done this before
I haven’t tried. Let me know if it works!
I am pretty much in the same situation. Did anyone try what @Bob_Carpenter suggested? Is somebody knowledgeable enough willing to try? I would do it but I do not even know where to start.
1 Like
To do this, you’d have to know how the Stan types actually translate to C++.
Theoretically, it’s possible to instantiate a bunch of templated functions and stash that as a library. We can’t do it for all, because it gets very expensive to do on disk. I tried to do that for the normal distribution a long time ago (just for double, stan::math::var, std::vector<double>, std::vector<stan::math::var>, Eigen::Matrix<double, -1, 1>, Eigen::Matrix<var, -1, 1>, Eigen::Matrix<double, 1, -1>, and Eigen::Matrix<var, 1, -1>) and that took something like 300+ MB.
I don’t think it’s ok to instantiate a handful only and keep that in a library, but I could be wrong.
But yes, it’s possible. You’d have to trace through which instantiations you need. For example, it might be quick to change priors if we had all the template instantiations for the likelihood all set up. But I’m not sure if that’s true.
A nice option for the future would be for stanc to dump out a file that made it easy to compile a library of only the instantiations in a given model.
@mitzimorris is going to start by refactoring the model class so that it can be compiled separately from the algorithms.
I am not sure what is your ultimate goal, but note that if you know all the possible alternatives beforehand you might be able to get close to 0 overhead for multiple model variants by using optional parameters/data (disclaimer, link is to my personal blog). The cost is however that code complexity increases a bit.
Thanks for linking about speeding up how to deal with multiple model variants. Sorry it’s such a hack now. This is also what rstanarm does under the hood.
We’re going after an orthogonal speedup—the basic compile time for the services, algorithms, and math lib.
The ultimate goal’s to not give users enough time to make an espresso between when a Stan model starts compiling and when it finishes. The problem now is that everything (the math lib and algorithms and services being used, specifically) gets recompiled every time when a model changes because everything’s templated for the model class.
2 Likes