Derivatives w.r.t data or hyperparameters for sensitivity analysis

It seems to me that derivatives of the density with respect to hyperparameters or data are quite useful for sensitivity analysis, and also should be pretty easy for Stan to provide automatically.

I looked around and found that Ryan Giordano has worked on this idea (at least for hyperparameters):
His implementation uses a clever trick to get hyperparameter derivatives out of the existing API. If I understand correctly, a user can write variable declarations in a hyperparameters block that are moved into transformed data to make a program for sampling, and then they’re moved into parameters in a second program to make them into autodiff variables.

There was also some discussion on Andrew’s blog, e.g.:

Is there a good reason this doesn’t seem to have gained traction?

If not, would there be interest in building this capability into the Stan language? I think the natural interface would be to support an annotation at the declaration of (global) variables. That way data variables could also be included.


I’m not sure, perhaps just lack of person hours to work on it? But I agree it seems like it would be useful!

1 Like

Sorry I missed your reply Jonah - glad I’m not missing something obvious, maybe I’ll look into it when I get some time.

1 Like