ROPE for ordinal regression model in brms

Interfaces brms
,
1

Hi everyone,

I am fitting cumulative models to analyse ordinal data with #brms. I want to set a region of practical equivalence (ROPE) and see if the highest density interval corresponding to my parameter is inside or outside this ROPE. However I can’t find guidelines for this. Does anyone has experience with this kind of approach (cumulative models + ROPE) ? How should we interpret ordinal regression parameters obtained with brms and what would be a good way to set a ROPE in this context ? @matti @Solomon @paul.buerkner maybe ?

  1. I would like to specify the rope depending on my variables (simplified example) :
  • Predictor → pseudo-continuous transformed as a z-score → range ~ [-2.5 ; 2.5]
  • DV → ordinal measure : 9 possibles values (from 1 to 9)

With the following reasoning :
a) range of predictor = 5.
b) smaller observable variation on the DV = 1
c) Smallest Effect size of interest (SESOI) = 1/5 = 0.2
d) SESOI/2 = 0.1 (we keep a safety margin because parameter values between 0.1 and 0.2 could potentially be of interest)
e) ROPE = -0.1, 0.1

However I am not sure of this kind of reasoning with an ordinal model.

  1. If the solution 1 is not possible, what about “classical” ROPEs ?

From Region of Practical Equivalence (ROPE) we can read :

"Kruschke (2018) suggests that the ROPE could be set, by default, to a range from -0.1 to 0.1 of a standardized parameter (negligible effect size according to Cohen, 1988).

  • For linear models (lm) , this can be generalised to:

[−0.1∗SDy,0.1∗SDy]

*-For logistic models , the parameters expressed in log odds ratio can be converted to standardized difference through the formula:

π/√3

(see the effectsize package, resulting in a range of -0.18 to -0.18 . For other models with binary outcome, it is strongly recommended to manually specify the rope argument. Currently, the same default is applied that for logistic models."

I would be tented to choose the second option because of the logistic nature of the ordinal model, but if I read well Bürkner & Vuorre (2019) or Understanding the disc parameter in ordered logit models - #5 by matti / https://groups.google.com/g/brms-users/c/SdXC3T9U9hU, the parameters given by brms for ordinal models are standardised parameters, and then we should choose the first option ?

Thanks a lot !

Brice

ref :

Bürkner, P.-C., & Vuorre, M. (2019). Ordinal Regression Models in Psychology: A Tutorial. Advances in Methods and Practices in Psychological Science , 2 (1), 77–101. https://doi.org/10.1177/2515245918823199

1 Like
2

Hi @Brice

The regression parameters of the ordinal model won’t be on the numerical scale of your DV, so thinking e.g. in terms of “b) smaller observable variation on the DV = 1” can be a little misleading. The regression parameters indicate effects on the scale of the continuous latent variable. So if your link function is probit, then the effects are in standard deviations. That’s what Kruschke is describing; cohen’s d is an effect size in standard deviations. So if you use a probit link function you can think about them in terms of cohen’s d, but with the difference that you are modelling a latent variable whereas cohen’s d is typically on an observed gaussian variable. Similarly, if you use a logistic link function, about .18 (or is it .16 I can’t remember) would be roughly similar in size.

In our tutorial manuscript, if I recall correctly, we used the probit link function because in psychology we are generally talking about gaussian variables and so there is a clear connection (probit link function = latent gaussian variable). Logistic behaves a little better in estimation though, and is the default.

3 Likes
3

I’m a fan of the probit link for the reasons @matti described.

1 Like
4

It’s also helpful to transform effects on the latent scale to their implied effects for the ordinal category probabilities, e. g. what is the probability change for each category for a one unit change in the predictor? This might be easier to set a ROPE for.

2 Likes
5

thanks for the information.

6

Thanks a lot !

Ok, I see. This works if I choose to set my ROPE depending on benchmarks.

I’m going to try to find a solution for an approach more specific to my variables.

Thanks again !

Brice

8

Hello all – I found this conversation in trying to solve this problem myself. Brice what was your solution?

9

Hi,

Sorry I am new to the Bayesian community. I just fitted an ordinal model following the tutorial provided by Bürkner & Vuorre using a probit link function.
Now I want to check whether some parameters fall within the ROPE using the BayesiantestR package. Is it correct for me to set the ROPE to -0.1 0.1 as the ordinal model expresses parameter estimates on the latent linear varibale or do I need to use the 0.18?

Thanks for your help!

Cheers,
Max

10

You can use any values for the limits of the ROPE that are meaningful for you and your research community. There are no universal defaults.

11

Yes, definitely I just wanted to use values that are usually thought of as too small to matter (Cohen)
So I was unsure whether the ordinal brms object requires -0.1 to 0.1 rope range or whether the parameter (because there is a probit link function in the background) needs the logistic conversion to -0.18 to 018

12

I don’t think I have enough information about your model to know how to comment further about the two ranges you’ve proposed. E.g., I don’t know anything about your predictor(s). It might help if you showed your code.

13 
bay_moving_cond_stim_id_id <- brm(Rating ~ Condition + stimulus_type + (1|ID)+ (1|image_id) , data=moving_long, family=cumulative("probit", threshold = "flexible"), chains = 5,
                                  iter = 3000, warmup = 1000, cores = 5)

The predictors both encode 2 differences in the condition of the experiment:
Condition: was it seen (condition A) or was it imagined (condition B)
stimulus_type: was the seen/imagined stimulus a FACE or an ARTWORK
Rating is the dependent variable and is how pleasurable the experience was (likert 1-7)

14

If both your predictors are categorical, then brm() will effectively use them like dummy variables. Given your use of the probit link, that puts your \beta coefficients on what you might call a latent Cohen’s d metric.

1 Like