I gave last October a talk on this topic, but it seems the notebook was not public until now.
Summary of workflow for how many digits to report
- Run inference with some default number of iterations
- Check convergence diagnostics for all parameters
- Check that ESS is big enough for reliable convergence
diagnostics for all quantities of interest
- Look at the posterior for quantities of interest and decide how
many significant digits is reasonable taking into account the
posterior uncertainty (using SD, MAD, or tail quantiles)
- Check that MCSE is small enough for the desired accuracy of
reporting the posterior summaries for the quantities of
- If the accuracy is not sufficient, report less digits or run
- Halving MCSE requires quadrupling the number of iterations
(if CLT holds).
- Different quantities of interest have different MCSE and may
require different number of iterations for the desired accuracy.
- Some quantities of interest may have posterior distribution with
infinite variance, and then the ESS and MCSE are not defined for
the expectation. In such cases use, for example, median instead
of mean and mean absolute deviation (MAD) instead of standard
deviation. ESS and MCSE for (non-extreme) quantiles can be
derived from the (non-extreme) cumulative probabilities that
always have finite mean and variance.