ROPE for ordinal regression model in brms

Interfaces brms
,
1

Hi everyone,

I am fitting cumulative models to analyse ordinal data with #brms. I want to set a region of practical equivalence (ROPE) and see if the highest density interval corresponding to my parameter is inside or outside this ROPE. However I can’t find guidelines for this. Does anyone has experience with this kind of approach (cumulative models + ROPE) ? How should we interpret ordinal regression parameters obtained with brms and what would be a good way to set a ROPE in this context ? @matti @Solomon @paul.buerkner maybe ?

  1. I would like to specify the rope depending on my variables (simplified example) :
  • Predictor → pseudo-continuous transformed as a z-score → range ~ [-2.5 ; 2.5]
  • DV → ordinal measure : 9 possibles values (from 1 to 9)

With the following reasoning :
a) range of predictor = 5.
b) smaller observable variation on the DV = 1
c) Smallest Effect size of interest (SESOI) = 1/5 = 0.2
d) SESOI/2 = 0.1 (we keep a safety margin because parameter values between 0.1 and 0.2 could potentially be of interest)
e) ROPE = -0.1, 0.1

However I am not sure of this kind of reasoning with an ordinal model.

  1. If the solution 1 is not possible, what about “classical” ROPEs ?

From Region of Practical Equivalence (ROPE) we can read :

"Kruschke (2018) suggests that the ROPE could be set, by default, to a range from -0.1 to 0.1 of a standardized parameter (negligible effect size according to Cohen, 1988).

  • For linear models (lm) , this can be generalised to:

[−0.1∗SDy,0.1∗SDy]

*-For logistic models , the parameters expressed in log odds ratio can be converted to standardized difference through the formula:

π/√3

(see the effectsize package, resulting in a range of -0.18 to -0.18 . For other models with binary outcome, it is strongly recommended to manually specify the rope argument. Currently, the same default is applied that for logistic models."

I would be tented to choose the second option because of the logistic nature of the ordinal model, but if I read well Bürkner & Vuorre (2019) or Understanding the disc parameter in ordered logit models - #5 by matti / https://groups.google.com/g/brms-users/c/SdXC3T9U9hU, the parameters given by brms for ordinal models are standardised parameters, and then we should choose the first option ?

Thanks a lot !

Brice

ref :

Bürkner, P.-C., & Vuorre, M. (2019). Ordinal Regression Models in Psychology: A Tutorial. Advances in Methods and Practices in Psychological Science , 2 (1), 77–101. https://doi.org/10.1177/2515245918823199

1 Like
2

Hi @Brice

The regression parameters of the ordinal model won’t be on the numerical scale of your DV, so thinking e.g. in terms of “b) smaller observable variation on the DV = 1” can be a little misleading. The regression parameters indicate effects on the scale of the continuous latent variable. So if your link function is probit, then the effects are in standard deviations. That’s what Kruschke is describing; cohen’s d is an effect size in standard deviations. So if you use a probit link function you can think about them in terms of cohen’s d, but with the difference that you are modelling a latent variable whereas cohen’s d is typically on an observed gaussian variable. Similarly, if you use a logistic link function, about .18 (or is it .16 I can’t remember) would be roughly similar in size.

In our tutorial manuscript, if I recall correctly, we used the probit link function because in psychology we are generally talking about gaussian variables and so there is a clear connection (probit link function = latent gaussian variable). Logistic behaves a little better in estimation though, and is the default.

2 Likes
3

I’m a fan of the probit link for the reasons @matti described.

1 Like
4

It’s also helpful to transform effects on the latent scale to their implied effects for the ordinal category probabilities, e. g. what is the probability change for each category for a one unit change in the predictor? This might be easier to set a ROPE for.

2 Likes
5

thanks for the information.

6

Thanks a lot !

Ok, I see. This works if I choose to set my ROPE depending on benchmarks.

I’m going to try to find a solution for an approach more specific to my variables.

Thanks again !

Brice

8

Hello all – I found this conversation in trying to solve this problem myself. Brice what was your solution?