Help write an autodiff handbook

The branch issue-7/implicit-functions has the chapter on algebraic equations with two derivations: one that uses the implicit function theorem, and one that uses the more general adjoint method. You can check out this branch and then compile the AD-handbook to read the chapter. The cool thing about this approach (pedagogically) is that algebraic solver provides a very simple example of the adjoint method. The traditional case of ODEs is more complicated.

This post covers how we should differentiate the algebraic solver in Stan. @betanalpha, @vianeylb, and I review the ODE case in this manuscript and provide further references; I’d said the benefit of this review is that we do the derivations in great details. Other papers I like:

I have work on constrained optimization, but I haven’t shared this yet. It still needs refinement.

5 Likes