Divergent transitions and model averaging issues

Modeling
, ,
1

Hi,

i am currently having two issues using brms models in R.

1st: Divergent transitions.

i am having a lot of trouble getting rid of the last few divergent transitions on my models. I have 7 models in total, and i am investigating the the effects of fire severity on 45 species. Around 15/45 species will give a divergent transition for 1-2/7 models. it is rarely over 1-2 divergent transitions and the pairs and trace plots look good. In my latest run i increased the iterations to 20k, with a 5k warmup, and thinned by 15, in a hope that this might resolve my issues. it did not. I have played with the priors, changing the df and the scale parameters and the current setup of 4, 0, 1 works best. Does anyone have any ideas on other ways to resolve this issue, i am completely out of ideas? the code block below is an example of one of my models.

Model without any fire severity variable

withCallingHandlers({
modeln ← brm(
as.formula(paste(species_name, " ~
scale(distance_to_perimetre) +
scale(shannon_fire_size_diversity) +
scale(ndvi) + scale(terrain_ruggedness) + scale(elevation) +
factor(period) +
(1|location) + (1|landscape) + (1|Obs) + offset(log(total_deployment_days_period))")),
family = poisson,
data = species_data,
prior = c(
prior(student_t(4, 0, 1), class = “b”), # for fixed effects
prior(student_t(4, 0, 1), class = “sd”) # for random effect standard deviations
),
save_pars = save_pars(all = TRUE),
control = list(adapt_delta = 0.9999, max_treedepth = 15),
chains = 4, iter = 20000, warmup = 5000, thin = 15
)

2nd problem: Model averaging.

When using posterior_average function, I cannot give it a list (in my case a list of species). So the code works if i write it out manually. Does anyone know a work around to this?

As an example:
Let’s say you have a list of four models:
model_list = list(m1, m2, m3, m4)

This doesn’t work:
posterior_average(model_list)

This does work:
posterior_average(model_list[1], model_list[2],model_list[3],model_list[4])

appreciate anyone help on this.

2

Additionally, in relation to the first problem. If the trace and pairs plots are fine, the Rhat and ESS values look good. is there a case to ignore the small number of diverget transitions?

3

Three things:

  1. Having a few divergent transitions is not a big deal at all. This often happens in the warmup stage even for clean models. In future this might be even less of an issue when using Pathfinder to get the warmup started, but in any case I wouldn’t worry.

  2. Setting adapt_delta = 0.9999, max_treedepth = 15, and iter = 20000 is probably a bad idea, as it can lead you to spending all your time waiting for your simulations to finish. If R-hat is fine, I recommend sticking with the default settings for those three tuning parameters.

  3. If any concerns remain, I recommend doing a simulation experiment. Choose reasonable parameters for your model, simulate fake data, and then fit your model and check that you can recover the parameters (to approximately within the standard errors of your fitted model). This can be done more formally–we call it simulation-based calibration checking (http://www.stat.columbia.edu/~gelman/research/published/2211.02383.pdf)–but in practice I usually learn enough by just doing the fake-data simulation once or twice and checking that the parameter estimates and uncertainties make sense.

4

Hi Andrew,

thanks for the information.

  1. Ok that is great to hear. I note that Runtime warnings and convergence problems page states ‘a small number divergences without any pattern in their locations can be verified to be unimportant’ (Runtime warnings and convergence problems). Im assuming this means if the Rhat, ESS, Trace plots, and Pairs plots are ok, it is safe to proceed? is Pathfinder available yet?

  2. I was previously running 6000 iterations and a 3000 warmup. Which seems to worth the best. adapt_delta was set at 0.9999 over the default 0.8 as it reduced the number of divergent transitions.

thanks again for your help