We got into a conversation about what our wiki on versioning and compatibility means w.r.t. the Stan Math library. We brought it up at the meeting yesterday, but @syclik wasn’t around and so I wanted to summarize the consensus from that meeting online and perhaps engage Daniel and the others from the meeting here if there’s more to be worked out.
We all agree that we’re currently promising backwards compatibility (via semantic versioning) to users of the Stan language (see the Stan wiki, above). I think we are, to some extent, trying to provide something similar with the Math library, but it’s much less clear what the user API is in this case because it’s just a bunch of source code and makefiles. I will try to write out what I think we mostly agree we should support, though we didn’t talk about all of this in the meeting as we were focused on the specific issue at hand.
- Reverse mode automatic differentiation only (fwd and mix still need work and tests)
- Single-threaded only, except through
- Use only through one of the top-level headers (e.g.
stan/math/prim/mat.hpp, and not any of the lower ones like
- Nothing in any of the
metadirectories, which have historically changed constantly
- Using only our Makefiles
- Almost entirely to make the Stan language awesome. If we can easily provide for non-Stan users of the Math library, then we shall, but it should always be suborned to the Stan language needs.
As a practical test case for this policy, please consider this PR. Once that is merged, anyone starting a thread on which they want to perform AD operations will need to call an
init() function. This will not affect single-threaded users using our Makefiles, nor will affect those using
map_rect. @syclik reads the versioning wiki I think pretty reasonably to say that we’d need to bump the major version number for this, because “A change in a library breaks backward compatibility if a program that worked in the previous version no longer works the same way in the current version. For backward-compatibility breaking changes, the major version number is incremented.” Because a program could have been written in the past 6 months based on 2.18.0 that used the Math library with threads, and now that program would need to call the
init() function on its threads.
However, I would propose we change the wording in that wiki to reflect our actual policy. We have made numerous backwards-incompatible changes by that incredibly high standard, and we have issues out for bugfixes now that will break previously working programs that relied on bugs, for example. I propose the following wording:
A change in a library breaks backward compatibility if a program using only supported APIs that worked in the previous version no longer works the same way in the current version. For backward-compatibility breaking changes, the major version number is incremented.
I don’t think we should be tied to supporting all source code we have ever released.
I’m curious how people feel about this, so here’s a poll.
Keep the existing wording
Change the wording
 For the two 2.18 releases, you could have, for the first time, used the Math library with C++ threads (with or without interacting with
map_rect). We did not advertise that you could do this without
map_rect and it seems likely that no one has been using this since it is about 6 months old and we don’t have many math library users anyway.