ShinyStan 3.0 Alpha test

Hello Everyone,

@jonah and I have been working on an update for shinystan and we would be very happy if you could try it out and provide feedback. We replaced the entire back end for the shiny application and used modules to rebuild it so it will become easier to add / remove features in the future.

Updates shinystan application

  • All plots are now based on the bayesplot package
  • Warnings are displayed upon opening of the application
  • All plots come with captions to guide users through the diagnostics
  • Plots open on certain variables based on the worst performing parameters in terms of n_eff to help users troubleshoot their models
  • Plots can be customized and exported as reports (markdown generated pdf), plots (pdf) or .rds files
  • Version 2.5.0 can still be called by adding old_version = TRUE to any launch function
  • Shinystan now works with objects of class stanjm, stanmvreg and blavaan

Report generation

We added a functionality to autogenerate reports for your models. This can be done for the diagnostics, the parameter estimates or both. The report can print information concerning the n worst performing parameters in terms of n_eff, specific parameters via the pars argument or all parameters in your model. The function is called generate_report and can output pdf, html and word files. One of the aims is to make it as easy as possible to autogenerate an online supplementary file for your work. Please let us know if you have any ideas on how to further improve this.

Installing

We would be very happy if you could try it out and provide us with feedback. To install the alpha test version 3.0 you can use the following r code:

if (!require("devtools")) {
install.packages("devtools")
}
devtools::install_github("stan-dev/shinystan", ref = "v3-alpha", build_vignettes = TRUE)

Providing feedback and reporting bugs

There are several ways in which you can provide feedback, report bugs or request features:

Hoping to get your feedback,

Best,

Johan and Duco

10 Likes

Nice! Some general comments

  • It would be nice to be able to have some hotkey to cycle through the variables (or have some way to quickly zip through the parameters)
  • I’ve personally found the mcmc_parcoord plots very useful for looking at where divergences are happening. Some additional stuff from that vignette may be nice to automate
  • It would be cool if shinystan utilized the structure of brms and rstanarm a bit more. For instance a brmsfit object has the data for posterior checks already built into the model. Maybe if it’s a brmsfit object you could also have a tab for looking at conditional effects. brms can also return back draws from the priors and it would be nice to have a quick way to check priors for each variable against the computed posterior
7 Likes

I don’t think I have any new comments, but I’ll download the new version and let you know (I’ve been using the alpha for months and love it!). I shared this on Twitter so hopefully you’ll get some useful comments from there too. :)

3 Likes

Hey Steve,

Thanks so much for the feedback! These are some great suggestions.

  1. I’ll see if this can be done with an actual hotkey or if adding another button to cycle through the variables will need to do the trick.

  2. Would you like the parcoord stuff to be added to the reports? They can take quite some time to generate, so I could add an option to ignore or run them.

  3. This is a great idea and a very good use case to see if adding specific modules for specific objects works out well. Do you have an example of the type of plots you would like to look at?

Best, Duco

Thanks so much Lauren!

Looks great!

Some comments

  • How is the xlim selected for autocorrelation plot? It seems to show too many lags. You could use Geyer’s rule (available soon in posterior https://github.com/jgabry/posterior/issues/33)
  • There is check box with text “Show/Hide Caption”. If I click that, does it Show or Hide, or Show and Hide? Maybe it could have just text “Show Caption”?
  • I would show captions by default, as they are quite useful for someone learning Shinystan and diagnostics
  • Could change “Trace plots” to be the last in the list (on the left sidebar under “MCMC”) as they are often the least informative
  • “No parameters have an Rhat value above 1.1.” the new recommendation is 1.01
  • Add Bulk-ESS and Tail-ESS?
  • Add local and quantile efficiency plots?
  • “No parameters have an effective sample size less than 10% of the total sample size.” This reminds about the issue that not all variables are parameters. Also it would be good to write something like “…have an effective sample size for estimating the posterior mean less than…” to remind that effective sample size depends on the function, too (identity for mean).
  • In area plots, add option for vertical scaling (part of bayesplot already)
1 Like