# Above chance in a Bayesian model

Hello,

As a beginner in bayesian stats, I have the following question. In my model, I test how speakers of RP language discriminate a particular vowel contrast compared to speakers of CG language (reference level). Here’s my model’s formula:

``````fit <- brm(response ~  0  + Intercept + language + (1 | subject), data = test, family = bernoulli("logit"), prior = prior, sample_prior = "yes")
``````
``````Family: bernoulli
Formula: response ~ 0 + Intercept + language + (1 | subject)
Data: test (Number of observations: 320)
Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
total post-warmup draws = 4000

Group-Level Effects:
~subject (Number of levels: 20)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept)     0.87      0.30     0.42     1.58 1.00     1373     2192

Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept     -0.06      0.32    -0.68     0.58 1.00     1694     1964
languageRP     2.76      0.62     1.59     4.07 1.00     2503     1846

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
``````

Now, I want to do a hypothesis testing to find the evidence of above chance performance (i.e., > 50%) in the discrimination of the contrast by the two populations. I would like to know the code for this testing

Your help will be much appreciated!

This should do the trick:

``````hypothesis(
fit,
c(
"Intercept > 0", # for the reference group
"Intercept + languageRP > 0" # for the RP group
)
)
``````

I need to do two things. First, to find the evidence of `languageCG < languageRP`. For that, I used the following code in hypothesis testing `0 < languageRP`. Am I correct?

The other thing is the one I asked, that is, to find if each language group performed above chance. Does the code you provided show evidence for above chance performance, that is, performance of more than 50%? (as the forced choice task had two options). What if the chance score was, for example, 70%. How the code would change?

Many thanks!

Yes

Yes

You would now need to be checking if your effects are greater than logit(0.7) = 0.8473 . So `hypothesis(fit, "X > 0.8473")`. Note that logit(0.5) = 0, which is why you would be checking if the effect exceeds 0 if chance is 0.5.

1 Like

Fantastic, many thanks again!