There’s an outstanding bug in Eigen that Brad Bell (author of CppAD) found:
http://eigen.tuxfamily.org/bz/show_bug.cgi?id=1480
Turns out they drop zero value entries which isn’t kosher for autodiff, as the gradients might not be zero. Unless this gets fixed, we won’t be able to use their routines for autodiff. Brad suggests a patch in the issue based on runtime traits, which is something Eigen supports.
Do we run into the problem of vars being exactly zero with anything other than starting values and underflow?
There’s a pragmatic solution in section 2.3 here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.119.1392&rep=rep1&type=pdf
Not that I can think of. I mean it’s not measure zero because we use a finite set of floating point representations, but one in 2^64 is good enough for me.
I am currently avoiding the problem by setting input values that are zero, but not identically zero, to the smallest non-zero value possible; i.e.
https://en.cppreference.com/w/cpp/types/numeric_limits/min
This is a kludge and not guaranteed to work. I think I know how this can be fixed in Eigen, but I have many other things I am working on and do not have the time to do it myself. I am willing to help if someone else would like to fix it.
One alternative is to use complex step derivative approximation, since we can use complex number with eigen’s sparse solver.