Item Response Theory with guessing probability that differs across questions

  • Operating System: 10.12.6
  • brms Version: 2.10.0

Hello brms users,

I’ve just read @paul.buerkner 's “Bayesian Item Response Modeling in R with brms and Stan” and this has been so useful and helpful. I am in a situation in which I’d like to specify an 3PL model (3 parameter model) that includes a term for the guessing probability. Paul provides an example of such a model in the paper above (page 34, a model called “formula_va_3pl”). However, in the model he specifies, the guessing probability is constant across questions in the dataset. In my own data, some of the questions had only 2 options and some had 3, which means that the probability of getting a correct answer from random guessing varies across these questions. I am wondering if there is a way to specify this in a brms formula? Here is the 2 parameter version of the formula I am working on, now I just want to add the third ‘guessing’ parameter …

bf(correct ~ 0 + item + (1 | personID))

The data has a column for “correct” (binary), a column for “item” and a random intercept on the individual taking the test “personID”. There is also a column for “n_options” that lists the number of choices per question. The model above partially pools the personID variable but has a non-hierarchical structure/prior on item.

What I’d like to be able to do now is incorporate the probability of guessing each item, such that the value estimated for the personID random effect parameter will take on the meaning “ability above and beyond what is expected from guessing”. And, ideally, the fixed effect of item which will also mean, “difficulty controlling for its guessibility”. Any suggestions towards any of those two goals would be highly appreciated.

Thank you very much,
Brian

Hi Brian,

I discuss varying guessing probabilities in another paper: https://www.mdpi.com/2079-3200/8/1/5

1 Like