Prediction interval in meta-analysis with brms

Hi! I’m conducting a meta-analysis with brms and I want to compute the prediction interval for my average effect size.
This is my model:

mod0_prior <- c(
  prior(normal(0, 2), class = "b", coef = "Intercept"),
  prior(cauchy(0, 1), class = "sd")
)

fit <- brm(eff_size_g|se(eff_size_se_g) ~ 0 + Intercept + (1|study_meta/substudy),
           data = meta_clean,
           prior = mod0_prior)

Now I know that prediction interval in meta-analytic model are computed as: \hat{\mu} \pm 1.96\sqrt{\hat{\tau^2}+Var[\hat{\mu}]}. Reading online, with brms and my nested structure, the simple predict() approach should be fine, taking into account all variability sources.
Now, I’m struggling with these approaches:
1.

# Using Tidybayes

pred <- add_predicted_draws(meta_clean %>% select(substudy, eff_size_g, eff_size_se_g),
                            fit, re_formula = NA)

# Then summarising across all study:substudy using mean or median and 95% interval
# predicting a new study using an average study-level standard error
new_data_un <- tibble(study_meta = "newstudy",  eff_size_se_g = mean(meta_clean$eff_size_se_g))

Actual distributions are a little bit different, but the first approach returns me a strange bimodal posterior distribution. At the same time, the first approach sounds like the most reasonable for me.
What do you think?

1 Like

Hey there! I’m sorry your question fell through… Unfortunately I don’t know much about neither meta-analysis nor brms. I saw @ jsocolar come up with good thoughts the other day in this thread. Maybe he knows waht’s going on?

Cheers,
Max

Hopefully this Twitter thread can help you https://twitter.com/bmwiernik/status/1473306749906169858

In summary, to predict new study populations in a normal-normal model, this is the required code:

nd = data.frame(study = "new", sei = 0)
  
 brms::posterior_predict(object = model,
                          newdata = nd,
                          re_formula = NULL,
                          allow_new_levels = TRUE,
                          sample_new_levels = "gaussian")

where “model” is the meta-analysis model fitted with brms

1 Like