This is sort of a “higher-level” question, but I think there are a lot of people in this forum who might have stumbled upon the very same issue and could share their own experiences or insights!
So, I prepared a registered report (RR) which includes a section about sample size planning.
As I had an idea of the typicial sample size (from several similar previous studies) I firstly reported that. However, I have received the Editor’s feedback on the RR and the first comment asks me to perform a power analysis to inform and support my decision (which I think is a sensible idea).
Since I represented my research hypotheses (as different population-level coefficients) with a GLMM that will be estimated with Bayesian methods (using
brms), my first thought was "let’s simulate data and fit the model!" (akin to this excelent walkthrough).
However, as my model has several predictors and group-level effects over most of those predictors, each run is rather slow (~10hs). I know this because I have performed a parameter recovery analysis using previous studies’ sample size (to assert the feasibility of my planned analysis).
Thus, even if I test just a few sample sizes (e.g., 4) with a small simulation batch (
n_sim = 500) per sample size, this results in ~20000hs > 2 years!.
So, it seems like this path is a dead-end…
To my -probably limited- knowledge, there might be some non-exclusive alternatives (albeit non-ideal):
- Specify a simplified version of the model (e.g., with group-level effects over the model Intercept, or even removing group-level effects altogether?) that still captures my hypotheses, and perform a power analysis with simulations.
- Argument that it is unfeasible to perform a proper power analysis (from what I described above)
I have found a related question in this very forum, and I have looked into the book-chapter refered in the accepted answer, but wasn’t able to extract any information that could guide my decision on what to do.
What do you think? Has anyone else been in this situation?
Plase, share any comment/experience/suggestions (or even other forums where it might be more likely to get answers)!