There’s an outstanding bug in Eigen that Brad Bell (author of CppAD) found: http://eigen.tuxfamily.org/bz/show_bug.cgi?id=1480 Turns out they drop zero value entries which isn’t kosher for autodiff, as the gradients might not be zero. Unless this gets fixed, we won’t be able to use their routines…

Do we run into the problem of vars being exactly zero with anything other than starting values and underflow?

There’s a pragmatic solution in section 2.3 here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.119.1392&rep=rep1&type=pdf

[image] bgoodri: Do we run into the problem of vars being exactly zero with anything other than starting values and underflow? Not that I can think of. I mean it’s not measure zero because we use a finite set of floating point representations, but one in 2^64 is good enough for me.

I am currently avoiding the problem by setting input values that are zero, but not identically zero, to the smallest non-zero value possible; i.e. https://en.cppreference.com/w/cpp/types/numeric_limits/min This is a kludge and not guaranteed to work. I think I know how this can be fixed in Eigen, …

[image] Bob_Carpenter: Unless this gets fixed, we won’t be able to use their routines for autodiff One alternative is to use complex step derivative approximation , since we can use complex number with eigen’s sparse solver.

Sparse matrices in Eigen with autodiff

Developers

yizhang July 30, 2018, 12:48pm 6

One alternative is to use complex step derivative approximation, since we can use complex number with eigen’s sparse solver.

Topic		Replies	Views
Am I Leaking? [Eigen autodiff] Developers	12	2077	April 29, 2019
Surprising memory usage with eigendecomposition Algorithms	6	1094	March 12, 2019
Eigen-AD: Algorithmic Differentiation of the Eigen Library Developers math	15	4674	December 19, 2019
Sparse matrices in Stan Developers	12	2728	February 8, 2018
Refactoring math with more general signatures Developers math	7	1024	October 25, 2019

Sparse matrices in Eigen with autodiff

Related topics