Quick question hopefully. If this is documented somewhere and I’m just missing it lemme know.
I’ve been reading through the fwd mode autodiff code to figure out what it does. Does someone mind checking if I’m making sense here?
I think I get what regular
fvar<double> is and functions of this.
I want to be clear on
fvar<fvar<double>> (call it ffd) allows for up to two derivatives.
The elements are:
ffd.v.v – value of the variable
ffd.v.d – derivative of the variable with respect to something (call it a)
ffd.d.v – derivative of the variable with respect to something (call it b)
ffd.d.d – second derivatives
I think my biggest question is do ‘a’ and ‘b’ need to be the same thing? From looking around the code, I think not, and the fact that ffd.v.d need not equal ffd.d.v leads me to this conclusion as well.
So in math, if ‘v’ is my variable, then
fvar<fvar<double>> just gives me access to:
v dv/da dv/db d^2v/dadb
And if I pass v through a function ‘f’, then the
fvar<fvar<double>> that gets spit out gives me access to:
f df/da df/db d^2f/dadb
And if I’m using fvar<fvar> instead of double, then instead of these things being numbers I can call grad in the reverse mode autodiff and get the derivatives of all of these guys with respect to any of my parameters? (so if I have a parameter c, for instance):
df/dc d^2f/dadc d^2f/dbdc d^3f/dadbdc
Am I thinking about this clearly?