Discuss roadmap draft section on "Interface Package Architecture"

[seantalts]

Just breaking out the roadmap thread into a few other threads for each section. This thread is about this section:

@seantalts

Interface Package Architecture

This recommendation applies to all interfaces that link against our C++ code directly. The ideal interface architecture seems to split out the current functionality into 4 packages: analyze, fit, *stan, and visualize. We decided our group had no purview over the visualization packages as there aren’t many visualizations that aren’t model-specific, but we decided that visualization doesn’t belong in any of these core packages.

StanAnalyze

This package just contains pre-compiled binaries for the functions exposed in Stan services (functions like the effective sample size calculation, producing a fit summary, etc). This package will not have runtime requirements on any Stan C++ header files (so no Math library, Stan headers, etc), nor will it require a C++ compiler toolchain at runtime. This will enable folks doing similar work who would like to use our calculations to use ours directly in a lightweight fashion without creating their own implementation. These functions are basically all at the parameter-level, with a signature of something like std::vector<double *> → double.

StanFit

StanFit is responsible for performing inference, e.g. creating a StanFit object to contain the results of fitting a stan model, like posterior samples, given data and a properly installed C++ toolchain. This package will require all of the Stan headers to be packaged into it as it will compile custom Stan models on the fly. It will be responsible for compiling models, instantiating them with data, and running an inference algorithm on the instantiated model.

*Stan

This package ties the previous two packages together into a unified, easier-to-use interface similar to RStan and PyStan today, though without any visualization functionality. We’d like to import the functionality from both Analyze and Fit and provide the analyze functions as methods on a fit object that can apply the analyze function automatically across all of the parameters in the fit.

/cc @bgoodri @jonah @ariddell @bparbhu @ahartikainen

[ahartikainen]

Given that StanAnalyze is independent of the main library, I think (on Python) it could and should be independent package.

So on Python is should be pip installable.

Also, viz stuff is hard to make general, should we still have some page going through recommended plot types? Or atleast have some comments on general plot holes with mcmc (overplotting, usefulness of the traceplot)?

[seantalts]

Yeah, the hope that whoever brought up this idea had (was it Jonah?) was that StanAnalyze would be able to be used by PyMC3 and whoever else to do their analyses for effective sample size, rhat, etc without requiring a C++ toolchain or the Stan headers and all the other overhead of the full rstan or pystan package.

Viz stuff is hard! We basically decided that since the visualizations you want to show depend on the model you’re trying to fit, your audience, and the Stan team member currently advising you :P, that we won’t try to set guidelines for it across the org. So I think the idea would be that e.g. @betanalpha can have his own recommended plots that are different from Jonah’s or Ben’s, for example.