I was working on improving numerical stability of some functions. I have currently identified 7 functions where I can make some improvements to numerical stability.
These changes are in theory independent, but they share a common theme of improving numerics in a similar way.
What is the preferred format for PR submission? I want to make this convenient for the maintainers to review.
My thinking:
PROS of many small PRs:
- Cleaner separation of independent (albeit related) changes
- May easily reject some of the changes while accepting the others
CONS of many small PRs:
- May be more demanding for human attention to go through it
- Kinda feels like spamming
I would prefer the smaller PRs; to the extent that you’re worried about spamming, I think the easier thing to do is just wait until the first few are merged before opening more, rather than grouping in to larger patches
It’s good to check if they can also share a common helper function to avoid repeating code, in which case a shared PR could make more sense, so that the code part that is doing most of the numerics improvement is reviewed only once. I’ve grouped my recent numerics PRs based on set of distribution functions sharing common code which has been improved. Of course some ideas like reflection can be used for many functions, but which can not be put in a shared helper function.
@WardBrian, I think I’m finished now creating numerics PRs. Sorry that there are so many of them.
Interesting, my work partially overlaps with yours and uses a similar technique to solve the issue for the logits. :)
I will separate my work to more branches and not push the things you have already addressed or that conflict with your PRs at least until they are closed.
Commenting here that as I had focused on first order derivatives, there were several cases where I didn’t push the limits as far as second order derivatives seem to require. Great that you are testing also that part!