Evid.Ratio as Bayes factor value from hypothesis() function

Interfaces brms
1

It seems like the Evid.Ratio value is being interpreted as the Bayes factor (see 1 and 2 for examples).

Following the first link for a simple binomial sampling example, suppose we are interested in testing the null hypothesis of whether the true binomial proportion is greater than or equal to 0.6, and we obtained 13 out of 16 successes. Let say we obtained expert opinion and use a Beta(2, 2) prior on theta, the binomial proportion. The theoretical Bayes factor value should be 24:458.

d <- data.frame(s = 13, k = 16)

m0 <- bf(
  s | trials(k) ~ 0 + Intercept,
  family = binomial(link = "identity")
)

Prior <- set_prior("beta(2, 2)", class = "b", lb = 0, ub = 1)

m <- brm(
  formula = m0,
  prior = Prior,
  data = d,
  sample_prior = "yes",
  iter = 1e4, 
  cores = future::availableCores()
)

h <- hypothesis(m, "Intercept > 0.6")
h$hypothesis$Evid.Ratio

However, the Evid.Ratio is only about 12. Now based on a widely-used scale to interpret Bayes factors proposed by Kass and Raftery (1995), the Evid.Ratio would be moderate evidence in favor of the null. However, the actual Bayes factor value when we compute by definition, where the prior probability against the null (0.648) is the beta(2, 2) density integrated from 0 to 0.6, and the prior probability in favor of the null (0.352) is the beta(2, 2) density integrated from 0.6 to 1, is

h$hypothesis$Evid.Ratio / (0.352/0.648) # actual Bayes factor value in favor of H_0: theta > 0.6

This value is close to the theoretical value, which suggests strong evidence in favor of the null. One can argue they both are in favor, but the different magnitude can give rise to different interpretations according to different scales!

Furthermore, it seems like there are values computed internally that allow for the computation of the Bayes factor by definition, as seen below:

# computation of BF_01 by definition
(mean(samples$b_Intercept > 0.6) / mean(samples$b_Intercept < 0.6)) / (mean(prior_draws(m) > 0.6) / mean(prior_draws(m) < 0.6))

So my question is this: why is the computation by definition above not being reported as the Bayes factor over Evid.Ratio?

  • Operating System: Window x86_64, mingw32 (R version 4.2.2)
  • brms Version: 2.19.0
1 Like