I am struggling to understand should I report the results with random effects or not (re_formula = NULL or NA).
I have a population based data. Patients are nested in differently sized counties and I would like to report the result closest to the truth. N = 6000. I am interested in population-level effect only (country level).
An additive non-hierarchical model shows an effect with the narrowest CIs.
Hierarchical free intercept model (1 | county) is equal according to loo(), but shows the same effect with slightly wider confidence intervals. R2 CIs are overlapping with the previous model, however, the mean is 2% lower for the hierarchical model.
The effects of the previous models will have much wider confidence intervals while I set “re_formula = NULL” in conditional_effects(). But they still show significant effects.
Which results should I report? Or how should I decide?
PS! I also did some frequential statistics. R2 including random effects was 9% better compared to fixed effects only.