With forward mode only all you can get out are directional derivatives, so you can associate each component with a certain inner product of a derivative with vectors. And at higher-order those inner products mix many different orders of derivatives making things very confusing.
For example, consider a many-to-one function, f : R^{N} -> R. Evaluating the function with a fvar<double>
will require 2N inputs: the N function inputs, call them x, and the N sensitivities, call them u.
The fvar<double>
returned by the evaluation will contain two values. The first value will be
the function value, f(x)
, which can be thought of as 0th-order derivative, if you will. The second value will be the inner product <u, df/dx(x)>
.
For fvar<fvar<double>>
you’ll need 4N inputs, the input x and three vectors which I’ll call u, w, and z. The evaluation of the function then yields
ffd.v.v: f(x)
ffd.v.d: <u, df/dx(x)>
ffd.d.v: <w, df/dx(x)>
ffd.d.d: <z, df/dx(x)> + <u, d^{2}f / dx^2 * w>
.
In particular, ffd.d.d braids together first and second-order derivatives. This is related to the fact that the compositional algebra of higher-order Taylor expansions is wacky.
For more details check out Chapter 1 of the Nomad Manual, https://github.com/stan-dev/nomad/tree/master/manual, which you’ll have to clone and compile.