Optimize Jacobians and make transforms accessible from language

This came up most recently here:

We really need to

  1. make the underlying constraint transform code more efficient
    – custom vari types for Jacobians
    – custom vari types for transformed values
    – use sum to accumulator if not already getting an accumulator passed in

  2. expose to Stan language
    – unconstraining transforms
    – constraining inverse transforms
    – log Jacobian determinant of inverse transform
    – ?? compound inverse transform + Jaobian ??

Should we somehow return the log Jacobian determinant and leave it
to the user to increment the target (log density) or should we try to build
it in somehow and make the function with Jacobians only callable from transformed
parameters and model blocks?

This might be a good intro project to diving into reverse-mode
autodiff. It shouldn’t be hard other than the linear algebra
around the positive-definite matrix types, but there are lots of
transforms and we need at least three functions each, so that’s a
lot of plumbing. The transforms so far are identity, lower, upper,
lower+upper, simplex, ordered, pos_ordered, unit, cholesky_corr,
cholesky_cov, corr_matrix, cov_matrix, so that’s at least 36 new
Stan functions.

  • Bob