Hi all,
First off, please don’t read this as a statement blame or judgement of any
sort. In the past our code base was smaller, there were fewer contributors,
fewer users, and much less optimization happening with the code base. As
we’ve evolved as a team, we’ve added process to reduce the downstream
burden of maintenance.
Here’s a recent example of when it would have been useful to have tested
and documented code. I’m looking at Bob’s fix for math issue #370
https://github.com/stan-dev/math/issues/370. The pull request looks
reasonable: https://github.com/stan-dev/math/pull/392
The place where there’s a change in behavior is in
stan/math/prim/scal/fun/grad_reg_inc_beta.hpp, but the place where it’s
actually failing is in the distribution tests for beta_ccdf_log(). If you
look at the pull request, that function isn’t one of the ones that changed
at all.
stan/math/prim/scal/fun/grad_reg_inc_beta.hpp has almost no doc, especially
boundary conditions when this function is suppose to work and when it’s not
supposed to work. There are a few tests in test/unit/math/mix, but not in
prim, which it should have.
Over the last two days, I think I’ve spent 3 hours getting down to an
internal function that doesn’t have tests, that wasn’t touched, and now has
different behavior. (I’ve actually tried to tackle this issue in the past
and put probably a good 16 hours without succeeding due to differences in
behavior with functions that aren’t doced and tested. Bob’s much more
patient than I am.)
I won’t claim that having tests in place would have prevented this from
happening. But I will claim that it would have saved hours of time if they
were in place. And I’ll also claim that it would have taken much less time
to have generated the tests when the code was written.
If anyone has spare time, an immediate thing you can help with is writing
unit tests and documentation for the Math library for any functions that
don’t have tests and doc. For the time being, I’m going to continue to
suggest strongly that internal functions have tests.
Daniel