I am trying to use
brms to fit a three-level model to combine results from different experiments. Essentially, I hope to model the data like a mini-meta-analysis, i.e., a meta-analysis of my own experiments.
I had three experiments, with sample size varied from 20 to 50s. All three experiments shared a same 2 * 2 * 3 within-subject design, the dependent variable are reaction times and d’ in signal detection theory.
I tried to model the reaction times data by
shifted lognormal distribution, as this blogpost have suggested. However, because this blogpost only provided example script for two-level data, I also read this tutorial and learnt form it.
My code is as below
m1 <- df_test %>% dplyr::mutate(RT_sec = RT/1000) %>% # log RT in seconds dplyr::filter(ACC == 1) %>% # only correct trials brms::brm(RT_sec ~ ismatch*Identity*Valence + (ismatch*Identity*Valence | ExpID) + (ismatch*Identity*Valence | ExpID:Subject), family=shifted_lognormal(), data = ., chains = 4, control = list(adapt_delta = .95), iter = 4000, thin = 2, cores = parallel::detectCores(), backend = 'cmdstanr')
In which the
ExpID is the id of my experiments,
Subject is the id of participants.
Valence are three columns for the independent variables.
here is the test dataset: df_test.csv (1.3 MB)
After fitting the model (long waiting time), there were warnings:
Warning messages: 1: Parts of the model have not converged (some Rhats are > 1.05). Be careful when analysing the results! We recommend running more iterations and/or setting stronger priors. 2: There were 59 divergent transitions after warmup. Increasing adapt_delta above may help. See http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
Also, the posterior of the population-level parameters are more dispersed than the individual experiments (see below). This is odd because, if the three-level model worked like mini-meta-analysis, the population level parameters were estimated based on overall sample size and therefore should be more precise than those of individual experiments.
I suspect I used a wrong model, which cause both divergence problem and the dispersed posterior of population parameter.
One potential reason is that the distribution for the third level (experiment level) should be different from the second level (individual level), but I donot know how to specify different distributions for different levels.
- Operating System:
version R version 4.0.4 (2021-02-15)
os Ubuntu 20.04.2 LTS
system x86_64, linux-gnu
- brms Version: 2.14.4
Looking forward to suggestions.
Thanks in advance.