Our pull request backlog is growing alarmingly, especially in the math lib.
I think the main holdup is continuous integration. While I appreciate that it’s saving us from terrible bugs, it’s a huge obstacle for moving forward with anything.
The first problem is that it’s a very demanding set of tests that people can’t really afford to run locally. We run multiple platforms and compilers on lots of tests, incluing downstream tests with dependent packages. There are at least three big serves in play that devs need to wrangle: Travis, Jenkins, and GitHub. Then there’s cpplint and clang format, both of which are requirements. This is a big hurdle, but I haven’t see any way to cut it down while maintaining both code quality and everyone’s sanity.
The second problem is that we don’t have good general ways to test new autodiff functions. Each one’s a crap shoot as to how much a reviewer is going to demand in the way of testing. The
mdivide_right pull request in process now is a good example. The previous version had zero tests of the autodiff. Sebastian wrote a more efficient version that did less copying. Now Daniel wants tests. Fair enough. I tried to write those tests and there are 16 combinations of matrix/vector and double/autodiff arguments for the two functions of two arguments each (one argument has to be a matrix). I started the cut-and-paste war, then backed off into a metaprogram, but that’s such a pain that after two functors and four functions that were nowhere near there yet, I gave up and decided to rant on discourse for awhile instead.
The only way out of this I see is to buckle down and write a serious testing framework for multivariate autodiff. Something that’ll take arbitrary functions, double versions (perhaps with expected results in some cases), and test everything. I’ve done this with the univariate functions and know how to do it with higher-order functions, I just can’t find the few weeks of coding I’d need to do it. Instead, I’m trying to work on things like MPI and the language refactoring.
How are we going to find the time to do this?