Multiple comparison in sample size planning

General
1

Hello,

I’m planning an experiment, in which I will use Bayesian log-normal mixed-effect model for reaction time data. In my field (behavioral science) researchers require sample size planning to suppress HARKing even when using Bayesian statistics.

I have read the following three papers/blogs below and probably understand simulation-based sample size planning based on predicted effect size. My question is what kind of problems we have and how to solve them when planning multiple comparisons in simulation based sample size planning. For the criterion of the effect, we can use both Bayes factor and 95%CI (and other probabilistic criteria). Do someone know the problems and solve for sample size planning for multiple comparisons when using Bayes factor and 95%CI.

In frequentist paradigm, we can use Bonferroni correction in sample size planning. Bayesian statistics also has such method in sample size planning?

https://link.springer.com/article/10.1007/s42113-021-00125-y

https://julianquandt.com/post/power-analysis-by-data-simulation-in-r-part-iv/

https://solomonkurz.netlify.app/blog/bayesian-power-analysis-part-i/

Thank you for your kindness.

2

I believe the multiple comparison problem is specific to the frequentist paradigm, where the aim is to control the type-1 (i.e., false positive) error rate.

You may be interested in the projpred package for variable selection for regression models fit with brms or rstanarm.

3

I’m so sorry for replying late. The notification mail was categorized in spam box… And thank you very much for your replying. I did not much know about variable comparison. The package may solve my current problem. I learn it! Thank you very much for your helpful comment!