Fvar<var>

[sakrejda]

Just wanted to say the Nomad manual is magically clear. Thx.

[sakrejda]

So almost there, I’m not sure how to get the reference value for the test at the end of this snippet: a.d_.d_ … I think this is the “second gradient” from table 1.1 of the Nomad manual but I can’t convince myself of that yet.

  using stan::math::fvar;
  using boost::math::gamma_p;

  fvar<fvar<double> > x;
  x.val_.val_ = 0.5001;
  x.val_.d_ = 1.0;

  fvar<fvar<double> > y;
  y.val_.val_ = 1.0001;
  y.d_.val_ = 1.0;

  fvar<fvar<double> > a = gamma_p(x,y);

  EXPECT_FLOAT_EQ(0.40753385, a.d_.d_);

[betanalpha]

Fvars

With forward mode only all you can get out are directional derivatives, so you can associate each component with a certain inner product of a derivative with vectors. And at higher-order those inner products mix many different orders of derivatives making things very confusing. For example, consider a many-to-one function, f : R^{N} -> R. Evaluating the function with a fvar<double> will require 2N inputs: the N function inputs, call them x, and the N sensitivities, call them u. The fvar<doub…

[sakrejda]

So yes: [quote=“betanalpha, post:2, topic:379”]
ffd.d.d: <z, df/dx(x)> + <u, d^{2}f / dx^2 * w>.
[/quote]

[Bob_Carpenter]

Sure, but a function f:R -> R^N requires only a single call to forward mode. Reverse-mode computes rows of the Jacobian in one pass whereas forward mode computes columns (if you take the directional derivative to be axis aligned).

If you start nesting, such as fvar<fvar<double>> then you get second derivatives, but it’s almost always better to use fvar<var> instead.

You may want to read Giles’s introduction to autodiff in his matrix autodiff tech report if you liked Michael’s. It’s very clear if you think in matrices.