Hi everyone!

I’m analyzing a study in which participants received training for 20 items. Participants were randomly assigned to the experimental (Train = Y) or the control (Train = N) group. The 20 items had been tested 3 times: pretest (Time = T1), immediate posttest (Time = T2), and delayed posttest (Time = T3). Participants’ performance in each item were coded as “Incorrect”,“Partial”, or"Correct". Person covariates included participants’ gender, age, and ability. Person-by-item covariates included participants’ familiarity with and bias against each item.

```
> str(testdata)
tibble [3,960 × 11] (S3: tbl_df/tbl/data.frame)
$ ID : chr [1:3960] "k14" "k14" "k14" "k14" ...
$ Train: Factor w/ 2 levels "N","Y": 2 2 2 2 2 2 2 2 2 2 ...
$ Gender : Factor w/ 2 levels "F","M": 2 2 2 2 2 2 2 2 2 2 ...
$ Item : chr [1:3960] "Item1" "Item2" "Item3" "Item4" ...
$ Time : Factor w/ 3 levels "T1","T2","T3": 1 1 1 1 1 1 1 1 1 1 ...
$ Age : num [1:3960] 6.07 6.07 6.07 6.07 6.07 6.07 6.07 6.07 6.07 6.07 ...
$ Ability : num [1:3960] 6 6 6 6 6 6 6 6 6 6 ...
$ Familiarity : Factor w/ 2 levels "N","Y": 2 2 1 2 2 2 2 2 1 2 ...
$ Practice : Factor w/ 2 levels "N","Y": 1 1 1 1 1 1 1 1 2 1 ...
$ Bias : Factor w/ 2 levels "N","Y": 1 1 1 2 1 1 1 2 1 1 ...
$ Performance : Ord.factor w/ 3 levels "Incorrect"<"Partial"<"Correct": 3 1 ...
```

I had run an adjacent category model with category specific effects as follows:

```
prior.test2pl<-prior("normal(0,3)",class="sd",group="ID")+
prior("normal(0,3)",class="sd",group="Item")+
prior("normal(0,1)",class="sd",group="Item",dpar="disc")
bf.test2pl.1<-bf(Performance~Train*Gender*Time+Age+Ability+Familiarity+cs(Bias)+(0+Time|ID)+(1|i|Item), +disc~1+(1|i|Item))
bay.test2pl.1<-brm(
formula=bf.test2pl.1,
data=testdata,
family=brmsfamily("acat","logit"),
prior=prior.test2pl,
)
```

```
> parnames(bay.test2pl.1)
[1] "b_Intercept[1]" "b_Intercept[2]"
[3] "b_disc_Intercept" "b_TrainY"
[5] "b_GenderM" "b_TimeT2"
[7] "b_TimeT3" "b_Age"
[9] "b_Ability" "b_FamiliarityY"
[11] "b_TrainY:GenderM" "b_TrainY:TimeT2"
[13] "b_TrainY:TimeT3" "b_GenderM:TimeT2"
[15] "b_GenderM:TimeT3" "b_TrainY:GenderM:TimeT2"
[17] "b_TrainY:GenderM:TimeT3" "bcs_BiasY[1]"
[19] "bcs_BiasY[2]" "sd_ID__TimeT1"
[21] ...
```

I’ve some questions when conducting post hoc analyses with the model:

Q1: Can I use the hypothesis() function to test whether Familiarity increases the probability of Correct Performance in T2 in the Train group? If so, should I only include the related variables (i.e. TrainY, TimeT2, and FamiliarityY; hpF1), or should I add the intercepts (hpF2), or should I include all variables except TimeT3 (hpF3)? Which is correct (if any) and what’s the difference among them?

```
hpF<-c("hpF1"="plogis(b_TrainY+b_TimeT2+b_TrainY:TimeT2+b_FamiliarityY)>
plogis(b_TrainY+b_TimeT2+b_TrainY:TimeT2)",
"hpF2"="plogis(b_Intercept[1]+b_Intercept[2]+b_TrainY+b_TimeT2+b_TrainY:TimeT2+b_FamiliarityY)>
plogis(b_Intercept[1]+b_Intercept[2]+b_TrainY+b_TimeT2+b_TrainY:TimeT2)",
"hpF3"="plogis(b_Intercept[1]+b_Intercept[2]+b_disc_Intercept+b_TrainY+b_GenderM+b_TimeT2+b_Age+b_Ability+b_TrainY:GenderM+b_TrainY:TimeT2+b_GenderM:TimeT2+b_TrainY:GenderM:TimeT2+bcs_BiasY[1]+bcs_BiasY[2]+b_FamiliarityY)>
plogis(b_Intercept[1]+b_Intercept[2]+b_disc_Intercept+b_TrainY+b_GenderM+b_TimeT2+b_Age+b_Ability+b_TrainY:GenderM+b_TrainY:TimeT2+b_GenderM:TimeT2+b_TrainY:GenderM:TimeT2+bcs_BiasY[1]+bcs_BiasY[2])")
```

```
print(hypothesis(bay.test2pl.1, hpF, class = NULL),digits=3)
Hypothesis Tests for class :
Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Post.Prob Star
1 hpF1 0.000 0.000 0 0 1141.857 0.999 *
2 hpF2 0.000 0.000 0 0 0.186 0.157
3 hpF3 0.005 0.045 0 0 2.548 0.718
```

Q2: Can I test whether Bias (which has category specific effects) reduced the probability of Correct rather than Partial Performance? Should I only include Intercept[2] (which represents the threshold between Partial and Correct, hpB1)? Or should I include both Intercept[1] and Intercept[2] (hpB2)? I can do both with the hypothesis() function, but I’m confused about the meaning of these combinations.

```
hpB<-c("hpB1"="plogis(b_Intercept[2]+bcs_BiasY[2])<
plogis(b_Intercept[2])",
"hpB2"="plogis(b_Intercept[1]+b_Intercept[2]+bcs_BiasY[2])<
plogis(b_Intercept[1]+b_Intercept[2])",
"hpB3"="plogis(b_Intercept[1]+b_Intercept[2]+b_disc_Intercept+b_TrainY+b_GenderM+b_TimeT2+b_TimeT3+b_Age+b_Ability+b_FamiliarityY+b_TrainY:GenderM+b_TrainY:TimeT2+b_TrainY:TimeT3+b_GenderM:TimeT2+b_GenderM:TimeT3+b_TrainY:GenderM:TimeT2+b_TrainY:GenderM:TimeT3+bcs_BiasY[2])<
plogis(b_Intercept[1]+b_Intercept[2]+b_disc_Intercept+b_TrainY+b_GenderM+b_TimeT2+b_TimeT3+b_Age+b_Ability+b_FamiliarityY+b_TrainY:GenderM+b_TrainY:TimeT2+b_TrainY:TimeT3+b_GenderM:TimeT2+b_GenderM:TimeT3+b_TrainY:GenderM:TimeT2+b_TrainY:GenderM:TimeT3)")
```

```
print(hypothesis(bay.test2pl.1, hpB, class = NULL),digits=3)
Hypothesis Tests for class :
Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Post.Prob Star
1 hpB1 -0.468 0.414 -0.996 0 Inf 1.000 *
2 hpB2 -0.033 0.157 -0.109 0 11.270 0.918
3 hpB3 0.000 0.002 0.000 0 0.345 0.256
```

Q3: How can I test whether the effect of Ability (a continuous variable) differs in the (Train = Y) and (Train = N) groups?

Thanks for your attention.