Hi - this is a bit of a more stats than brms/stan specific question, but I haven’t received a response on typical stats forums, and I also wonder if this is something other people using brms have encountered this and understand what is going on:
I’ve been trying to conduct some power analyses/calibration for a hierarchical, ordinal regression. In a simple case a group is measured at Time 1, treated, and measured again afterwards at Time 2. I simulate data using mvnorm function in R, it simply lets you set a correlation between paired observations, and so you can specify e.g., that Time2 should be correlated with Time1, but .5 higher on a standard normal scale. To get the data into and ordinal form I place cutpoints along the normal distribution to get 5 responses (<-1.5, <-.5, <.5, <1.5, >= 1.5).
If I run a cumulative ordinal regression with a probit link simply as Time 1 vs. Time 2, it perfectly recovers these cutpoints and the effect size of .5. However, if I specify in the model that observations are nested within subjects - i.e., each time 1 and 2 observation is paired - there is an inflation in the estimates of both the cutpoints and the effect size (e.g., rather than .5, the effect size might be .7 or so), and this inflation increases with increasing correlations between time 1 and time 2.
Why is this occurring? Notably, it doesn’t occur if I run the same regression models not as an ordinal regression but simply on the normally distributed data, so I think it is something specific to the ordinal model, and not something generally for nested models.
Why does nesting the observations within subjects inflate the estimates? I’ve attached screenshots of the output with vs. without the within ppt nesting below.
