Interpreting brms results

I tried to nest idiomaticity(spatial/nonspatial usages) into test (pretest/posttest/finaltest), but the output didn’t show ‘DelayIdiom’, is it being treated as the reference category? I would like know how to fix this so that I can observe the learning effect for idiomatic usages on the delayed posttest as well. Thank you so much!

dat2$TestByIdiom.F = as.factor(ifelse(dat2$TestByIdiom == 0, ‘PreSpatial’,
ifelse(dat2$TestByIdiom == 1, ‘PreIdiom’,
ifelse(dat2$TestByIdiom == 2, ‘PostSpatial’,
ifelse(dat2$TestByIdiom == 3, ‘PostIdiom’,
ifelse(dat2$TestByIdiom == 4, ‘DelaySpatial’, ‘DelayIdiom’))))))

as.factor will set the order of the levels in alphabetical order, which in this case means that DelayIdiom will come first and therefore will be set as the reference level. You can set a different reference level by using the factor function and specifying the order of the levels with the levels argument, or you can use forcats::fct_relevel to set a new reference level without having to specify the order of all of the levels. Also, you can simplify recoding the value of TestByIdiom using either case_when or recode instead of nested ifelse statements. For example:

library(tidyverse)

# Recode values of TestByIdiom
dat2$TestByIdiom = recode(dat2$TestByIdiom,
                          "1"="PreIdiom",
                          "2"="PostSpatial",
                          "3"="Postidiom",
                          "4"="DelaySpatial",
                         .default="DelayIdiom")

# Convert TestByIdiom to a factor and set PreIdiom as the reference level
dat2$TestByIdiom = fct_relevel(dat2$TestByIdiom, "PreIdiom")

Regardless of which factor level is the reference level for any particular variable, you can use the hypothesis function to look at the modeled effect of various contrasts between different factor levels. For example, see here.

Thanks joels - this is very helpful!!

1 Like

Hi joels,

Thanks again for suggesting the hypothesis function. However, I am not sure if I computed correctly regarding the contrast:

If I want to know whether the schematic group performed better than the traditional group on idiomatic uses, is the following correct?

a1 ← c(a1 = “Condition.Fschematic:TestByIdiom.FPostIdiom + Condition.Fschematic:TestByIdiom.FPreIdiom – Condition.Ftraditional:TestByIdiomPreIdiom+Condition.Ftraditional:TestByIdiom.FPostIdiom > 0”)
hypothesis(brm.cloze_prior_newdata, a1)

Thank you so much in advance T.T

Family: bernoulli
Links: mu = logit
Formula: Response ~ Condition.F/TestByIdiom.F + (1 | Participant) + (1 | ItemNumber)
Data: datTrans (Number of observations: 15188)
Draws: 4 chains, each with iter = 3000; warmup = 1500; thin = 1;
total post-warmup draws = 6000

Group-Level Effects:
~ItemNumber (Number of levels: 36)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 1.43 0.21 1.07 1.88 1.00 672 1212

~Participant (Number of levels: 79)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 0.90 0.05 0.80 1.00 1.00 1337 2338

Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
Intercept 0.20 0.30 -0.38 0.79 1.01 485
Condition.Fcorrective -0.32 0.22 -0.76 0.10 1.01 1020
Condition.Fschematic 0.02 0.21 -0.39 0.41 1.01 917
Condition.Ftraditional -0.09 0.21 -0.51 0.32 1.00 891
Condition.Fcontrol:TestByIdiom.FDelayIdiom -0.47 0.30 -1.07 0.11 1.01 629
Condition.Fcorrective:TestByIdiom.FDelayIdiom -0.20 0.25 -0.69 0.28 1.02 392
Condition.Fschematic:TestByIdiom.FDelayIdiom 0.07 0.24 -0.41 0.54 1.02 399
Condition.Ftraditional:TestByIdiom.FDelayIdiom -0.12 0.24 -0.60 0.35 1.01 449
Condition.Fcontrol:TestByIdiom.FDelayspatial 0.23 0.16 -0.09 0.54 1.00 5958
Condition.Fcorrective:TestByIdiom.FDelayspatial 0.78 0.12 0.56 1.01 1.00 7012
Condition.Fschematic:TestByIdiom.FDelayspatial 1.10 0.11 0.88 1.31 1.00 6744
Condition.Ftraditional:TestByIdiom.FDelayspatial 0.84 0.12 0.62 1.07 1.00 6586
Condition.Fcontrol:TestByIdiom.FPostIdiom -0.82 0.25 -1.32 -0.32 1.02 443
Condition.Fcorrective:TestByIdiom.FPostIdiom -0.12 0.24 -0.60 0.35 1.02 403
Condition.Fschematic:TestByIdiom.FPostIdiom 0.13 0.23 -0.35 0.57 1.02 384
Condition.Ftraditional:TestByIdiom.FPostIdiom 0.31 0.24 -0.16 0.77 1.01 409
Condition.Fcontrol:TestByIdiom.FPostspatial -0.19 0.13 -0.45 0.06 1.00 5315
Condition.Fcorrective:TestByIdiom.FPostspatial 0.96 0.11 0.75 1.17 1.00 5888
Condition.Fschematic:TestByIdiom.FPostspatial 1.19 0.10 1.00 1.39 1.00 6678
Condition.Ftraditional:TestByIdiom.FPostspatial 1.00 0.11 0.79 1.21 1.00 5683
Condition.Fcontrol:TestByIdiom.FPreIdiom -0.97 0.26 -1.49 -0.47 1.01 463
Condition.Fcorrective:TestByIdiom.FPreIdiom -0.89 0.24 -1.37 -0.42 1.02 380
Condition.Fschematic:TestByIdiom.FPreIdiom -0.87 0.23 -1.33 -0.42 1.02 373
Condition.Ftraditional:TestByIdiom.FPreIdiom -0.49 0.24 -0.96 -0.02 1.02 427