# Help in reply to reviewer regarding CI

**URL:** <https://discourse.mc-stan.org/t/help-in-reply-to-reviewer-regarding-ci/33416>\
**Category:** General\
**Created:** [November 24, 2023, 6:15am UTC](https://discourse.mc-stan.org/t/help-in-reply-to-reviewer-regarding-ci/33416 "2023-11-24T06:15:09Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![JohnKruschke](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/johnkruschke/32/21245_2.png) [@JohnKruschke](https://discourse.mc-stan.org/u/JohnKruschke)\
**Post date:** [November 26, 2023, 2:56pm UTC](https://discourse.mc-stan.org/t/help-in-reply-to-reviewer-regarding-ci/33416/5 "2023-11-26T14:56:09Z")

</div>

The _Bayesian Analysis Reporting Guidelines_ (BARG) are intended to be helpful in cases like this. They’re published open-access here: [https://www.nature.com/articles/s41562-021-01177-7](https://www.nature.com/articles/s41562-021-01177-7) (Disclosure: I wrote the article.)

The [BARG](https://www.nature.com/articles/s41562-021-01177-7) provide guidance for **reporting decisions** (among all other components of an analysis). First, do you really _need_ to make decisions, or is the decision a ritual? If you do need to state decisions, then the BARG provide advice either using a _credible_ interval (CI) or using posterior model probability.

- If using a CI, the [BARG](https://www.nature.com/articles/s41562-021-01177-7) are agnostic about what size of interval (or type of interval, HDI vs ETI) to use. The BARG emphasize that CI limits computed from MCMC are very wobbly because the limits are (typically) in low-density regions of posterior distribution; therefore high effective-sample size (ESS) is required. I think to more deeply justify the size of a decision interval you’d have to go full-blown decision theory with specified costs and benefits; for initial thoughts about doing that see the [_Supplement_](https://osf.io/fchdr) (to a [different open-access article](https://journals.sagepub.com/doi/full/10.1177/2515245918771304)) available here: [https://osf.io/fchdr](https://osf.io/fchdr). Meanwhile, in practice, I’d say to go with whatever size of interval you think is most useful for your research _and your audience_; after all, if they’re not convinced by a weaker decision criterion (i.e., only 89% instead of 95%) then your research will have less impact. I agree with [zhez67373](https://discourse.mc-stan.org/u/zhez67373) regarding one possible option to satisfy both: put the audience-preferred criterion in the article and put your preferred criterion is supplemental material.

- If using a Bayesian hypothesis test via model comparison, the [BARG](https://www.nature.com/articles/s41562-021-01177-7) recommend reporting posterior model probabilities (_not_ only Bayes factors, BFs) and basing a decision on the posterior model probability exceeding a criterion. Notice you need to specify a decision criterion for the posterior model probability; should it be 95%? 89%? Something else? A key issue here is that the posterior model probabilities depend on the assumed prior model probabilities. Therefore, the BARG recommend reporting the posterior model probabilities for a range of prior model probabilities, and, in particular, **reporting the minimum prior probability the model could have and still meet the decision threshold**. This concept is illustrated by [Figure 2](https://www.nature.com/articles/s41562-021-01177-7#Fig2) in the BARG. (All of this assumes you have used appropriate prior distributions on parameters within the models; see the separate section in the BARG regarding prior sensitivity analysis.)

There is lots more discussion in the [BARG](https://www.nature.com/articles/s41562-021-01177-7).

P.S. I gave a [talk about this at StanCon 2023](https://mc-stan.org/events/stancon2023/#invited-speakers).

---

_[View the full topic](https://discourse.mc-stan.org/t/help-in-reply-to-reviewer-regarding-ci/33416)._