I estimated an ordinal IRT model following the steps outlined in Bürkner (2020) here https://arxiv.org/pdf/1905.09501.pdf. In doing so, it seems that brms estimates a person’s latent trait score even when they have answered fewer items than the total number of items. That is, an IRT score is estimated even when a person has answered, for example, only 3 of 8 items. Does anyone have insight into how brms does this? Are the missing items just dropped from the estimated equation, meaning this person’s score is just based on the 3 items they did answer?
Thanks in advance.
There isn’t anything specific on the brms side that causes you to be able to estimate a theta score from partial data; instead, that’s a feature of the IRT parameters.
In long format, each row corresponds to an item response observation, and by default, brms will omit any rows where there are missing data. It could be that data for the responses are missing, or if you are fitting expanded IRT models where there are predictors of theta or the item parameters, then missing predictors will also be excluded by row. Using all the remaining rows, brms will estimate the parameters from the observed data.
In the case of IRT, a person’s ability is a product of their responses to items and those items parameters. When we are able to know something about an item’s parameters, then we are also able to learn something about a person’s ability level based on how they responded to those items. Thus, we don’t actually have to know a person’s responses to all the items to know about where their ability level is; instead, we just need to know the item parameters well-enough and some of their responses to items to start making decent guesses. You should observe that the posterior estimate of the random effect (i.e., your theta estimate) is more uncertain (has higher variability) for those individuals with fewer observed item responses since, while we have some information as to where their ability level is, we will always be more confident of what a person’s ability is when we have more observations from them. The fact that we can get decent guesses of latent ability from partial item responses is commonly used in IRT models with missing data
1 Like