Sparse matrices in Eigen with autodiff


There’s an outstanding bug in Eigen that Brad Bell (author of CppAD) found:

Turns out they drop zero value entries which isn’t kosher for autodiff, as the gradients might not be zero. Unless this gets fixed, we won’t be able to use their routines for autodiff. Brad suggests a patch in the issue based on runtime traits, which is something Eigen supports.


Do we run into the problem of vars being exactly zero with anything other than starting values and underflow?


There’s a pragmatic solution in section 2.3 here:


Not that I can think of. I mean it’s not measure zero because we use a finite set of floating point representations, but one in 2^64 is good enough for me.


I am currently avoiding the problem by setting input values that are zero, but not identically zero, to the smallest non-zero value possible; i.e.

This is a kludge and not guaranteed to work. I think I know how this can be fixed in Eigen, but I have many other things I am working on and do not have the time to do it myself. I am willing to help if someone else would like to fix it.