Using a “black box” likelihood function

I have a model/likelihood truely is a “black box” then we can just use the good-old-fashioned finite difference to find the gradients.
In pymc3, I am able to do this, see https://docs.pymc.io/notebooks/blackbox_external_likelihood.html.
Would it be possible in stan as well?

Thanks.

For what it’s worth, I asked a similar question a couple years back: https://discourse.mc-stan.org/t/possible-catches-in-interfacing-stan-with-black-box-software/

Thank you for your reply. Have you ever succeed implementing something as they suggested?

I never needed to myself.

It’s not built-in, if that’s what you’re asking. Might be interesting to do that at some point.

External functions can and have been added to Stan, but it requires non-trivial back end C++ plumbing. Only non-trivial in that it’s a lot of work—it’s all known unknowns (other than finite diff precision).

Finite differences may be old, but I wouldn’t characterize them as good! They’re very slow, requiring on the order of 1 (point and point + epsilon) 2 (point - e, point + e) or 4 (point, point - 2e, point - e, point + e, point + 2e) function evaluations per dimension to use the centered or more stable double versions (as we use, for example, in our testing framework).

They also tend to be low accuracy and unstable near zero. In some cases, you can get enough accuracy in the derivatives to make them usable for HMC. We don’t really know where that boundary is—it certainly depends on the model’s curvature.

Other approaches are to do someting like build an approximation to the black box function, for example using a Gaussian process.

Thank you for the detailed explanation. I will discover the possibility to approximate my function as Gaussian process. In the my time, I can still use Random Walk Metropolis method to do inference, which does not require the gradients. Does Stan implement this method?

No, because we’ve never been able to figure out how to automatically tune proposals to make it work in a reasonable amount of time.

You might want to look at something like the emcee package in Python, which is designed for these kinds of black box proposals and should be better than random-walk Metropolis.