Bayes factors is inconsistent with 95%CI

Interfaces brms
1

Hi here,
I using hypothesis() function to get bayes factors BF01 (H0 against H1), however sometime I found that the result of 95%CI would inconsistent with BF01. As shown in following picture, BF01 is smaller than 1 when the 95%CI was excluded 0 in most time，which is a consistent context that both of them support H1 and against H0 (i.e., row 2,4,7,8). However, sometime I found BF01 is inconsistent with 95%CI, as shown in row 16, the 95%CI [-0.30, -0.02] is excluded zero, but BF01 was larger than 1, which confuse me that 95%CI is prefer H1 and BF01 is prefer H0. It is thankful if someone could help me understand why this conflict happens.

  1. List item
    Hypothesis Tests for class b:
    Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Post.Prob Star
    1 (groupAccpriming) = 0 -0.11 0.11 -0.34 0.11 5.43 0.84
    2 (groupMoralpriming) = 0 0.30 0.11 0.07 0.52 0.35 0.26 *
    3 (rating_accuracy.z) = 0 0.06 0.04 -0.02 0.14 7.84 0.89
    4 (rating_morality.z) = 0 0.24 0.07 0.10 0.38 0.02 0.02 *
    5 (rating_difficult… = 0 -0.03 0.04 -0.11 0.06 18.56 0.95
    6 (rating_familiari… = 0 0.05 0.04 -0.04 0.13 12.60 0.93
    7 (rating_importanc… = 0 0.28 0.05 0.18 0.38 0.00 0.00 *
    8 (groupAccpriming:… = 0 0.34 0.06 0.23 0.46 0.00 0.00 *
    9 (groupMoralprimin… = 0 -0.05 0.06 -0.17 0.06 11.00 0.92
    10 (groupAccpriming:… = 0 -0.09 0.08 -0.24 0.06 6.29 0.86
    11 (groupMoralprimin… = 0 -0.01 0.08 -0.17 0.15 13.11 0.93
    12 (groupAccpriming:… = 0 0.08 0.06 -0.04 0.19 7.43 0.88
    13 (groupMoralprimin… = 0 -0.05 0.06 -0.17 0.06 11.67 0.92
    14 (groupAccpriming:… = 0 -0.03 0.06 -0.15 0.08 15.17 0.94
    15 (groupMoralprimin… = 0 -0.06 0.06 -0.17 0.06 10.29 0.91
    16 (groupAccpriming:… = 0 -0.16 0.07 -0.30 -0.02 1.40 0.58 *
    17 (groupMoralprimin… = 0 -0.02 0.07 -0.16 0.12 12.95 0.93

‘CI’: 90%-CI for one-sided and 95%-CI for two-sided hypotheses.
‘*’: For one-sided hypotheses, the posterior probability exceeds 95%;
for two-sided hypotheses, the value tested against lies outside the 95%-CI.
Posterior probabilities of point hypotheses assume equal prior probabilities.

2

These two metrics are not ‘measuring’/indicating the same thing and so it is possible that they can seemingly suggest different things. The 95% CI is indicating that there is relatively low posterior probability at 0.

The 95% HDI focuses only on summarising the posterior distribution.

The Bayes Factor is like a ratio of evidence, comparing the likelihood of the data under a null hypothesis relative to the likelihood of the data under an alternative hypothesis.

If the data is not very far from 0, then it is totally possible that this is still considered most consistent with no real difference between conditions underlying the observed ‘apparent’ difference. In short, the two are not measures of the exact same thing and so, while they often will point to a similar conclusion - especially for large differences - this need not be the case.

1 Like
3

Thank~ I got what you mean. This result may happens when the upper or lower boundary is very close to zero (like row 16, -0.02 very close to zero), so BF give a conservative decision, which still support H0 even if 95%CI exclude zero. It is very nice to know that this conflict maybe happen sometime and the result did not cause by the mistake I make.