I conducted a study using Multilevel Regression and Poststratification (MRP). One reviewer raised the following concern:
“Following Abadie et al. (2023), if the authors insist on conducting the analysis at the province level, then the standard errors should be clustered at the same level. I am not sure whether the authors are doing this correctly, and, if not, how this affects the main results.”
How should I respond to this comment?
Abadie et al. (2023) is a frequentist paper, is it not? You might point out that if you computed the Bayesian posterior distribution at the province level, the posterior \textit{SD} will already be calibrated for the province level. No further adjustment needed.
I think we’d need more context.
What does this mean? I’m not even sure what “conducting the analysis at the province level” means. What analysis? Does this imply only analyzing at the province level (e.g., no hierarchical modeling or sharing of strength through empirical Bayes)?
I used MRP to conduct small-area estimation and measured province-level opinion
What are they objecting to? If you’re modeling at the province-level, you should be providing uncertainty at that level, too. Of course if you’re doing this in a Bayesian framework, you want to report credible intervals rather than confidence intervals. The Monte Carlo standard errors aren’t so relevant in a Bayesian analysis—they don’t correspond to the standard error of a frequentist estimator.