Thank you for your insight, @Bob_Carpenter !
I appreciate your confirming my understanding and setting of the ROPE range and suggesting that I avoid testing hypotheses dichotomously. I totally agree with you, and the reason I got so fascinated by Bayesian modeling in the first place was that I thought that I could interpret results in a more nuanced manner.
However, I am having difficulty dealing with the practical aspect of analysis in my research area to take full advantage of the posterior. If I may, I would like to ask a follow-up question about this.
Could you kindly let me know which reference (and the section name and/or page number, if possible) you would recommend I read into this? I have read Bayesian Data Analysis (Third edition) as well as other Bayesian books (Statistical Rethinking and Doing Bayesian Data Analysis), but I feel that I am not establishing the way I want to interpret the posterior distribution. However, I could be missing a lot.
In my research area, most studies still rely on the frequentist approach. So, even if Bayesian modeling is adopted, the “meaningfulness” of a predictor variable is judged simply based on whether or not the 95% Credible Interval includes 0.
Many people recommend setting a credible interval range based on the area’s knowledge, but there is no practical shared idea on the size of credible intervals in the area. Also, if we keep judging whether the credible intervals include zero or not, I feel that we cannot avoid the binary decision… This is probably because we need to (1) set a range that sounds familiar to the majority (i.e., 95%) and (2) judge the meaningfulness of predictors quickly based on a pre-determined range (either 95% or 89% or anything else, but decided beforehand to make ‘subjective’ judgment) because usually there are many predictors to examine in a study.
Could you suggest how to deal with such a situation?
I may be missing, but I feel that I have to have a pre-determined cut-off point to make a judgment. I could make a statement such as “the probability of a participant providing an accurate response on the final test being above 80% was 80%”, based on the posterior… but this seems to be fairly complicated as talking about the probability of the probability, so does not seem so appealing to reviewers.
I could describe the posterior with multiple ranges (e.g., 5%, 10%, 50%, 90%, and 95% CrI), but I am unsure how to make a nuanced judgment without a predetermined cut-off range, such as whether 90% CrI excludes 0…
I have been reading many articles and books on hypothesis testing in Bayesian and learning about the probability of direction and ROPE, and other stuff (e.g., Reporting Guidelines • bayestestR), but I feel I am still having difficulty taking the best advantage of bayesian modeling.
Any suggestions and references that I can learn and read into would be very much appreciated!