Help with ArviZ dashboard

General
1

Hi, we have started building a dashboard package for Bayesian inference. Dashboarding is new to us so any help/input from the Stan community is welcomed.

Dashboards are build on top of the great Panel package. To increase interactivity, we use Bokeh plots.

There are couple goals

  • general customizable inference dashboard
  • posterior predictive dashboard with widgets
  • loo (reloo) dashboard
  • streaming interface (e.g. coupled with PyStan3/CmdStanPy)

See

So, just to reiterate, any help is welcomed, also opinions about what we should add and do.

5 Likes
2

This seems more ambitious than ShinyStan, and that is pretty useful. I think if I had any complaint about shinystan is it is hard to handle boatloads of parameters if you’re dealing with dropdown menus sorted in some unexpected way. Not sure if there’s a way around that though.

Does PyStan3 have an official asynchronous API? I would like one for cmdstanX.

1 Like
3

Yeah, I was thinking some kind of regex / search with option to “select” a subset. That said, going over huge amount of visualizations is something that needs a good interface.

PyStan3 gets draws async. There is not yet a way to access these draws middle of the sampling, but I think this is something we could add in future.
For CmdStan I was thinking somekind of tail / watch solution, until IO will update.

These of course assume we have a good idea what parts are needed for pleasent experience. (e.g. TensorBoard use is seamless with tensorflow use).

4

Yeah I wonder about some sort of iterative thing.

Like you make certain types of plots for lots of parameters, and then you widdle it down to smaller types of plots.

I made a shiny app the other day that I’m pretty happy with: https://bbbales2.shinyapps.io/math-shiny-perf/

To use it, first you look at a table which has summary information so you can quickly find the thing you care about. Secondly, you scroll down and select plots of that actual thing.

Doing that for diagnostics with posterior draws makes sense. First find the ones with low ESS/bad rhat and then do plots of them, so maybe that sort of iterative approach would be useful.

2 Likes