Hypothesis testing, continuous predictors

Interfaces brms
,
1

Hi!

I have a model where I am trying to predict var_1 by the interaction between var_2 and var_3. I would like to test hypotheses regarding how var_1 is predicted based on different levels of var_2 and var_3. All variables are continuous.

For example, in this conditional_effects plot, it seems that at high levels of var_3, the relationship between var_1 and var_2 is positive, while for low levels of var_3, the relationship between var_1 and var_2 is negative.

000016

Is it possible to test this in a hypothesis? A method I’ve seen online is to multiply the interaction term by the SD, as below. But I don’t really understand how this works with continuous variables. Is the hypothesis testing the average level of var_2 at different levels of var_3, or the slope of the interaction between var_2 and var_3 at different levels of var_3?

hypothesis(model_h1, "(var_2:var_3*3.48)
                          <
                          (var_2:var_3*2.37)",
           alpha = 0.025)

I would appreciate any feedback or ideas!

2

I can’t help with the use of hypothesis(), but since you welcome any feedback, if I was dealing with this I would do something like this:

library(emmeans)
trends = emtrends(m,
                  var = "var_2",
                  specs = "var_3", 
                  at = list(var_3 = c(2.37, 3.48)))
summary(trends)
pairs(trends)

emtrends() will give you the slopes of the var_1 ~ var_2 relationship at specified values of var_3. From a nhst standpoint, is the credible interval at var_3=2.37 clearly positive? Is the other one clearly negative?
Then pairs gives you the difference of the two slopes. Is that one clearly positive?

You could also use bayestestR::p_direction()

p_direction(emtrends(m,
                     var = "var_2",
                     specs = "var_3", 
                     at = list(var_3 = c(2.37, 3.48))))

p_direction(pairs(trends))
1 Like
3

Thanks so much for the idea! Coincidentally, since I posted the question I have also looked into how to run the contrasts with emtrends, so this is very useful!

Do you have any experience with ROPE and bayes factor obtained from emtrends comparisons?

I have less than 5% of pd inside ROPE, which would mean that I can reject the null value, but at the same time I have a very small BF (<0.1), which if I understand the bayestestR output correctly, this would mean that the mean is 0. These two don’t seem to go hand in hand, so I must be mistaken somewhere.

For reference, I followed the example here: https://discourse.mc-stan.org/t/getting-bfs-for-specific-contrasts-in-categorical-or-ordinal-models/23093

4

Well, are your priors flat? And how did you choose your ROPE?

5

Priors are set with auto_prior and for the ROPE I used the default range from the bayestestR package, which sets it to x +- 0.1*SD(response) .

6

Default priors are flat so BF would be meaningless. The default ROPE assumes standardized variables.

For what it’s worth, I don’t think you need any of that. What I proposed earlier was not meant as an intermediate step; I would stop there.

2 Likes
7

That makes sense. Thank you for taking the time and answering my question!

1 Like