Multidimensional IRT in brms

Dear Paul,
in your recent excellent paper “Bayesian Item Response Modeling with brms and Stan”, you said that “the non-linear multilevel formula syntax of brms allows for a flexible yet concise specification of multidimensional IRT models”. Could you please give a short example of brms code how to specify a multidimensional model for polytomous data with estimated discrimination? Eg. 10 items in total, 5 of them in one dimension and another 5 in othoer dimension. Thanks a lot. Martin

Please also provide the following information in addition to your question:

  • Operating System: Windows
  • brms Version: 2.11.1

Hi Martin,

welcome to the Stan forums. Under your circumstances, the 1PL model could look as follows:

bf(y ~ (1 | item) + (0 + itemtype | person),
   family = bernoulli())

where itemtype is a factor assigning the items to the two dimensions. The corresponding 2PL model, could look as follows:

bf(y ~ disc * eta,
   eta ~ (1 | item) + (0 + itemtype | person),
   disc ~ (1 | item),
   nl = TRUE,
   family = bernoulli()
)
1 Like

Dear Paul,
thanks a lot for your reply. Unfortunately, my items are polytomous, as I said previously, so the family will be “cumulative” (or graded response model in IRT terminology). As you have said: “We have to use slightly different formula syntax, though, as the non-linear syntax of brms cannot handle the ordinal thresholds in the way that is required when adding discrimination parameters.” Your example: “formula_va_ord_2pl <- bf(resp ~ 1 + (1 |i| item) + (1 | id), disc ~ 1 + (1 |i| item))”. Should I add (0+itemtype) to (1|id) to obtain (0 + itemtype | id)?

Yes, exactly. The key is to replace (1 | id) by (0 + itemtype | id).

I have some experimental data where respondents saw a random political slogan and then reported how it made them feel on a 0-10 scale across 10 emotions which theory suggests load onto three latent factors.

They look as follows:

# A tibble: 94,530 x 7
      id    w8 gender slogan          emotion      resp  group          
   <dbl> <dbl> <fct>  <fct>           <fct>        <ord> <fct>          
 1 97322  1.07 Female Get Brexit done Enthusiastic 5     Positive Affect
 2 97322  1.07 Female Get Brexit done Hopeful      10    Positive Affect
 3 97322  1.07 Female Get Brexit done Proud        5     Positive Affect
 4 97322  1.07 Female Get Brexit done Scared       5     Anxiety        
 5 97322  1.07 Female Get Brexit done Worried      5     Anxiety        
 6 97322  1.07 Female Get Brexit done Afraid       6     Anxiety        
 7 97322  1.07 Female Get Brexit done Hateful      5     Aversion       
 8 97322  1.07 Female Get Brexit done Angry        5     Aversion       
 9 97322  1.07 Female Get Brexit done Bitter       5     Aversion       
10 97322  1.07 Female Get Brexit done Resentful    5     Aversion

I’d like to model how varying the slogan and respondent gender affects their latent emotions.

I fit the following model to a single slogan, which seemed to work well:

brm(
  formula =
    bf(resp | weights(w8) ~ 1 + (1 |i| emotion) + (0 + group | id),
       disc ~ 1 + (1 |i| emotion)),
  family = 
    cumulative(
      link = "probit",
      link_disc = "log"),
  prior = 
    prior(normal(0, 1), class = "Intercept") +
    prior(constant(1), class = "sd", group = "id") +
    prior(normal(0, 3), class = "sd", group = "emotion") +
    prior(normal(0, 1), class = "sd", dpar = "disc") +
    prior(lkj(2), class = "cor"),
  data = 
    dta %>%
    filter(slogan == "Get Brexit done"),
  iter = 2e3,
  chains = 2,
  cores = 2
 )

My question: How can I extend the model to handle covariates like gender that might themselves vary over the slogan that each respondent saw?

You could add, for instance, slogan*gender to the predictor term. Or, if you want to use multlevel structure over slogans, add (gender | slogan) instead.

Is there any way to have these vary over group? I.e. as if I were predicting their effect on the 3 different latent variables? (here shown as group).

They could interact with group of course.

Sorry for all the questions, this is all a little mind-bending — I assume that I also wouldn’t include a main effect in that case.

For example:

bf(resp ~ 1 + (1 |i| emotion) + (0 + group*gender + group*slogan | id),
   disc ~ 1 + (1 |i| emotion))

When adding varying effects you almost always want to include the corresponding overall effects.

Ok, thanks. I’m just trying to wrap my head around why there’s no overall effect of group/itemtype in the example you gave above. Was this an oversight?

Yeah I missed that sorry.

1 Like

Ok, cool! Good to know that I wasn’t going crazy!