Thanks, @apoorvreddy, @dmuck, and @hyunji.moon! I’m happy you’re on board.
I’m guessing you’re ok with me laying out what I think we should do; I’d like to turn that around and say that I do like things being collaborations, so we can start that way, but I’d really like your vision, input, and feedback so we can do things to accomplish that.
The first practical milestone I’d like to accomplish is to have a set of C++ tests that instantiate the algorithms:
- standalone tests. I’d like the tests to have as minimal dependencies as possible. For me, this means that we’ll rely on the
stan-dev/stan repo and the
stan-dev/math repo, but hopefully won’t need the
- initially test at the API boundary. We’ll use what we expect CmdStan to call. So the boundary should be the
- Enough breadth to have simple cases and more complicated cases. As I’m thinking about what we want to reimplement, I know I’d want tests of single iteration, multiple iterations, warmup only, post-warmup only, and different models.
- continuous integration. Have this run through GitHub actions if possible.
- documented. At least have it documented where we can all get to the repo, understand what we’re trying to accomplish, and how to run the tests.
- (optional) write up how to instantiate at this point. I think getting this far would be valuable for the community to know. But it’s optional. At least we’d have 3 additional people who would be able to instantiate the algorithms.
I think that’s actually going to take us a bit of effort. And just to be clear, at this point, we wouldn’t have done anything other than setup.
From there, I think it’s up to us how to proceed and which algorithms to tackle first. I think the optimization may be the easiest to reimplement, but we may choose to do something else first.
Just so we’re on the same page, I am thinking of first doing a proper “refactoring.” If that’s a new term, please see Code refactoring - Wikipedia. That means we leave the behavior the same and it’s a pure implementation change. We will be tempted to change the behavior, but that’s something different. Taking this approach of separating out the refactoring step from changing the behavior makes it a lot easier to tell that we’re doing the right thing. We have to match the current output exactly in order for us to know that we’ve made sound changes.
I’d like the work on this to be out in the community, so let’s continue to use Discourse. We’ll use GitHub issues as an immediate todo. And if we do have meetings, we can post any important discussions back here.
I’ll dm you to see when you’re free for a video call! (I don’t even know what time zones everyone is in!)