Thanks!

The best place to start is to send a note here to discourse or to one of the devs once you have an issue you think you want to work on or if you’re looking for something to work on. In addition to all this C++, we have tons of R and Python and other interface language work to do.

The closest thing to this is the JStatSoft paper.

There’s also the algorithms part of the manual that describes roughly how the MCMC and adaptation and variational inference and optimization work. As well as diagnostic mode. But it’s all pretty incomplete.

Most of our algorithms compute a log density or log likelihood depending on model and philosophical perspective. All our useful algorithms use gradients. Some also need Hessians and Riemannian HMC uses third-order derivatives (though in a compound form that is only quadratic to compute).

For full Bayes, we run MCMC, but we also have an optimizer for max penalized likelihood and posterior mode Laplace approximations, and also variational inference for approximate Bayes.

Stan’s default MCMC is an evolved form of NUTS. We’ve replaced the slice sampling with a mulitnomial sampling and compute acceptance rates and adaptation a bit differently. But the bottom line is that it’s not a Metropolis algorithm like static HMC that proposes then accepts or rejects. It’s more like it proposes a range of points including the starting point then selects one in a way that’s biased away from the starting point. That’s why it can be so efficient. That’s all covered in the NUTS paper, but fair warning, it’s dense. Michael Betancourt’s exhaustive HMC paper on arXiv is the best theory for all of this.

WinBUGS could dispaly graphical models visually as graphs. It even had a graph editor for building models with a GUI. But it was the 1990s and the results weren’t very pretty. The ones from Greta work much better.

Stan’s not a graphical modeling language, though, so you can’t produce such graphs. We could produce parse trees, but that’s different. Even the expression graph can vary based on iteration (like Pyro, unlike Edward or PyMC3, etc.)