Brms to estimate variance components for ordinal measured response

Interfaces brms
1

Dear Community,

Thank you Paul (brms author) for referring me to this forum.

I am doing a reliability study, using the methods of https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on using brms to estimate and use of the variance components.

Background: I have 20 subjects, 2 fixed raters, 2 testing sessions, and 10 trials per sessions. my dependent variable is a continuous, interval, variable (scale 0-10) [ subjective pain rating]. So participants are asked to provide a whole number between 0-10, repeatedly. Sessions are nested within each subject-assessor combination, and trials nested within each session. I want to estimate the variance components, in itself, but also to calculate an ICC to produce a relative measure of inter-rater and inter-session reliability .

My model is:

brms_mod = brm (vas ~ RATER +
(1|SUBJ) +
(1|SUBJ:SESSION) ,
data = df %>% filter (SIDE == “R”),
family = cumulative(“logit”),
threshold = “equidistant”)

My question is: in lmer, the inter-trial variance (level 1) is found in the residual. But in brms, no residual variance is produced. Am I right? Having done some reading (https://www.ncbi.nlm.nih.gov/pubmed/20569253), the level 1 variance is fixed for ordinal responses. Does this mean that (1) I cannot quantify inter-trial variance (e.g. using 1|SUBJ:SESSION:TRIAL) because it is fixed; (2) or can I quantify inter-trial variance?

Thank you for the kind help.
Kind regards,
Bernard

1 Like
2

I cannot really help you, as I am not intimate with brms, but you might improve your chance of getting a reply by moving this into the “brms” category (it is now categorized as “General”)

3

Although I haven’t used ordered logits, my understanding is that the level-1 variance can be assumed to be fixed at 3.29, allowing you to add that to denominator for calculation of VPC’s.

See that and other methods here:
https://www.tandfonline.com/doi/abs/10.1207/S15328031US0104_02