Is check_hmc_diagnostics the quickest and simplest way for a newbie to verify convergence?

Modeling
,
1

I’m once again having to fit and compare hundreds of categorical multilevel models using brms with the cmdstanr backend, and the computational cost is enormous. Due to that cost, I need to minimize computation time by using as few iterations (and as low an adapt_delta) as possible. These minimization needs make it imperative that I have some quick and simple metric for checking whether the fit is healthy.

What’s the simplest and quickest way for an inexpert to verify that the model has converged properly?

Can I rest assured that whenever brms does not produce automatic warnings, it is safe to proceed? This would be the most convenient approach.

Or should I rather do something like all(rhat(model)) < 1.05 and if the answer is TRUE, consider it safe to proceed?

Or would it be better to call rstan::check_hmc_diagnostics(model$fit), and assume it’s safe to proceed whenever this check reports zero divergences, zero iterations saturating the maximum treedepth, and zero “pathological” E-BFMI behaviors?

2

I run check_hmc_diagnostics and make sure that \hat{R}< 1.01. Of course posterior checks then tell me something about the fit.

1 Like