I saw this work about speeding up sampling from models which use latent guassian variables using adjoint-differentiated Laplace approximation from some members of the stan dev team recently
Multivariate probit models augment the ordinal data with guassian latent variables and are quite slow in Stan, so this would be really useful. I use bgoodri’s parameterization of multivariate probit (+ extended to ordered multivariate probit and hierarchical, based on the same techniques) - would it be possible to code this model using the the adjoint-differentiated Laplace approximation?
Although the latter mentions EP in the title, it has useful information on practicality of Laplace approximation for Multinomial probit.
So it has been done before in other software, but probably requires non-small amount of work to be implemented in Stan. May also require sparse matrices to be fast.
Thanks, i’ll have a look at these. By the way, would this method also speed up sampling for models with large number of observations, but without high-dimensional matrices, like some other proposed algorithms (e.g. calibrated Gibbs, transport monte carlo, etc) do? or do they only offer benefit for models which use big correlation matrices?
As @avehtari said, we’re in the process of extending our prototype to accomodate more likelihoods. At this point, you’d need to do some coding in C++. But I’ll add the multivariate probit on our todo list.
A quick follow up: we’re well on track to build a prototype that allows users to specify their own likelihood. So we might not explicitly construct a function for multivariate probit, but you should be able to specify it yourself.
Under the hood, we’re doing some careful autodiff and preliminary tests suggest that, with this more general implementation, the performance hit is not too bad (in fact, autodiff even seems to be faster than using analytical derivatives).