I often have to do power calculation and sample size estimation for biological experiments.
usually the biologist is interested in how many mice they need in their experimental setup to detect some effect size with some power. They may or may not have some prior data from which I can infer the expected measurement error or effect sizes.
Recently I shift more and more to Bayesian models to analyse their data (usually with brms) but now struggle to answer the sample size question in the bayesian framework.
When doing these in the frequentist framework, I usually simulate data under different experimental setups (different effect sizes, different number of animals per experimental unit). I use the prior information to simulate these data. In each simulated dataset I do a frequentist test (eg. testing contrasts using lm or lmer) and then calculate the proportion of simulations that pass this test (eg significant under 5% p value) . This is the power. If I can say to to biologist that they need eg. 4 animals per group to have 80% power to see a certain effect size of interest, they are happy.
Now I want to achieve something similar under the bayesian framework. My first thought was to again simulate data many times, do a bayesian analysis and define some criteria like have 80% of the posterior distribution of the parameter of interest be above some effect size threshold. And then again calculate the proportion of “passed” tests.
I have a feeling that this not the optimal way to go about it. For one, doing eg. 1000 bayesian analyses with mcmc on the different simulated datasets will take ages.
So if I want to know how many animals I need to observe a certain effect size with a certain confidence, how would I do this in the Bayesian framework?
I have seen this previous post:
And also read the material reference in the last answer, but does not really click with me.
What would be very helpful if somebody would now of an example sample size analysis with eg. brms on some real (or synthetic) dataset that I can tease apart. I had no luck in finding any.
Thanks for any pointers you can give ! :)