Just wanted to say the Nomad manual is magically clear. Thx.


So almost there, I’m not sure how to get the reference value for the test at the end of this snippet: a.d_.d_ … I think this is the “second gradient” from table 1.1 of the Nomad manual but I can’t convince myself of that yet.

  using stan::math::fvar;
  using boost::math::gamma_p;

  fvar<fvar<double> > x;
  x.val_.val_ = 0.5001;
  x.val_.d_ = 1.0;

  fvar<fvar<double> > y;
  y.val_.val_ = 1.0001;
  y.d_.val_ = 1.0;

  fvar<fvar<double> > a = gamma_p(x,y);

  EXPECT_FLOAT_EQ(0.40753385, a.d_.d_);

So yes: [quote=“betanalpha, post:2, topic:379”]
ffd.d.d: <z, df/dx(x)> + <u, d^{2}f / dx^2 * w>.

Sure, but a function f:R -> R^N requires only a single call to forward mode. Reverse-mode computes rows of the Jacobian in one pass whereas forward mode computes columns (if you take the directional derivative to be axis aligned).

If you start nesting, such as fvar<fvar<double>> then you get second derivatives, but it’s almost always better to use fvar<var> instead.

You may want to read Giles’s introduction to autodiff in his matrix autodiff tech report if you liked Michael’s. It’s very clear if you think in matrices.