@andrewgelman and @avehtari introduced HMC and variational inference as “1.5. Generic computation algorithms” in their paper on great statistical ideas of the past 50 years. Might the word “generic” explain backend’s superb computing (optimizing) ability to sample from joint distribution without explicit “information on how to simulate non-modelled variables”?
I wish to carefully ask for feedback on using the word estimator instead of backend. For two reasons.
First, while communicating modelers outside Stan community, I felt estimator is easier to communicate (which I am open to any objection!). The context was this paper which uses Stan’s HMC estimator but generator is defined using system dynamics simulation language in an implicit manner. Authors of the paper developed a package which translates the simulation language to differential equation functions in stan (tagging @charlesm93 who might be interested and can give good advice). The paper lists two justification for using Stan: faster estimation and visual diagnostics. SBC, however, is not included in their workflow which I am in the process of proposing. For instance, diagnostics section from the notebook in appendix is a good place to start.
Second, definition of suffix -or is “one that does a (specified) thing”. generator and estimator pair exists in both mathematical and program version. Data generating process for generator, computational algorithm for estimator will be the mathematical version while their implementation such as generator d For me, backend is program version of estimator and hence does not correspond well with generator compared to estimator.
In summary, I wish to get feedback on
i) the following frame, with example in the table
ii) using generator and estimator in a narrow sense (only for program)
| mathematical | program | |
|---|---|---|
| generator | Poisson regression statistical model | Poisson regression simulator (SBC package) |
| estimator | Hamiltonian differential equation system | HMC backend in C++ (Stan package) |
In general, mathematical version of generator would be convolution of prior and likelihood while estimator may be the form of
\begin{aligned} \hat{\nu}(\theta) &=\arg \min _{\nu} L(\nu ; \pi, x) \\ &=\frac{\exp \{-l(\theta, x)\} \pi(\theta)}{\int \exp \{-l(\theta, x)\} \pi(\mathrm{d} \theta)} \end{aligned} suggested in this paper which introduce variational formulation of Bayesian update.