Problems with low ESS when using brms::brm_multiple

Modeling
,
1

Hi,
I am experiencing very low ESS when using brm_multiple (some values in the region of 100 when I would expect 2000). Here are details of relevant information, I would be really grateful for some advice. Thank you.

  • Windows 10, R version 4.1.3, BRMS version 2.16.3, MICE version 3.14.0

  • Data with 2700 observations has approximately 14% missing data, all of class numerical. Used mice to create 30 imputed datasets using method=“cart”. Plots used to assess quality of imputations and I was satisfied.

  • Data used in a brm_multiple model with one random effect and 40 fixed effects, horseshoe priors applied to 30 of the fixed effects, default priors applied otherwise. Used 2000 iterations (1000 warmup), 4 chains. Specified a high target acceptance criterion of 0.98.

  • If I select one of the imputed datasets and run as a brm model, the ESS and rhats are all satisfactory. However, if I run the brm_multiple model i.e. to pool all 30 imputed datasets, I am left with very low ESS as described above (NB also convergence warnings and rhats>1.05 but as described in the brm_multiple documentation this is likely false warnings due to the chains across different datasets maybe not overlapping)

I am grateful for any advice, thank you.

1 Like
2

I have encountered the same issue! But if I look at the individual models using the code provided here, the bulk/tail ESS are much higher than the summary output of brm_multiple suggests (false positive). I think it has to do with the way how brm_multiple computes the pooled ESS and they are not representative of the individual ones like the Rhats. @paul.buerkner How does brm_multiple compute pooled ESS’s and do you have an idea why it is super low sometimes despite the individual ones being high?

3

The overall ESS are misleading. In the latest brms versions (at least from github), I am no longer showing them in the summary output. Instead, I present a bit of code in the doc of brm_multiple how to compute convergence measures individually for each model.

2 Likes