Ordinal regression -- exploring an interaction

Interfaces brms
1

Apologies for this newbie question. I’ve discovered the Bürkner & Vuorre (2019) paper and am quite besotted with the technique. I just need to work up the learning curve!

I have entered an interaction term into our model (see below; see too a graph plotted from the data). I am wondering how to explore the interaction. I feel there are several options but would appreciate your views on the legalities of these options:

  1. Via a similar post, I feel running ‘simple-effects’ style analysis on the separate levels of one of our effects is one way to do this
  2. I look to the generated Credible Intervals (see the graph) and report which are overlapping, which are not (and hence, which have evidence for being statistically different).
  3. I would also like to know the magnitude of difference between the 4 conditions. I wonder if I can rerun the analysis but, instead of having 2 conditions of ‘sound_type’ and 2 of ‘sound_output’ I create a combined factor of sound with 4 levels. By strategically shifting the baseline ‘level’ I can then show the magnitude of difference with the other levels (in terms of standard deviation). Note that the left hand graph would be the ‘baseline’ (exploratory analysis, not hypothesised but an interesting and intuitive finding).

May I thank-you for your help,
with best wishes,
Andy.

fit_sa1 <- brm(formula =dv ~ 1 + sound_type + sound_output + sound_type * sound_output , data = mydata2, family = cumulative("probit"))

image
image781×523 53.9 KB

Family: cumulative 
  Links: mu = probit; disc = identity 
Formula: sound_memory ~ 1 + app + output + app * output 
   Data: mydata2 (Number of observations: 153) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Population-Level Effects: 
                           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept[1]                  -1.71      0.25    -2.21    -1.25 1.00     2724     2619
Intercept[2]                  -1.45      0.22    -1.89    -1.03 1.00     3096     2864
Intercept[3]                  -0.24      0.18    -0.59     0.11 1.00     2998     2924
Intercept[4]                   0.79      0.18     0.43     1.16 1.00     2848     2900
apprecorded                    0.57      0.24     0.07     1.03 1.00     2219     2432
outputspeakers                 0.67      0.25     0.19     1.16 1.00     2299     2710
apprecorded:outputspeakers    -1.00      0.36    -1.71    -0.31 1.00     2040     2487

Family Specific Parameters: 
     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
disc     1.00      0.00     1.00     1.00   NA       NA       NA

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
2

With regards to your second option, I wouldn’t compare the CIs by overlap. Rather, I’d compute a formal contrast and summarize that contrast by its central tendency and CI.

1 Like
3

Many thanks! I’ll explore this :)

UPDATE: For future readers. This post is really helpful: Matti’s homepage - How to calculate contrasts from a fitted brms model

1 Like
4

Matti is a wellspring of knowledge

1 Like