From @seantalts:
https://j-towns.github.io/2017/06/12/A-new-trick.html
a new trick: a method for calculating jvps by composing two reverse mode vjps! This could render specialised code for forward mode redundant.
OK, so I took a look. Their “jvps” is a Jacobian-vector product. We need forward-mode for higher-order autodiff. And not just Hessian-vector products (what that GitHub repo calls “hvp”).
The author concludes that if you’re doing reverse mode the way we are, then
This makes our new trick, in its basic form, significantly less efficient than the forward mode implementation that I’ve written.
We can do the Hessian-vector products in the usual way, of course: