Hi, I am trying to do some kind of post-hoc analysis for each fraction of data.
Here is the graph, and what I want to report is that each color line’s slope in the three graphs are all different from zero.
I am wondering in which part of the modeling result I should look into to know this.
formula_full = p_negativeR ~ SS + FT + SS* FT + (1|SID)
# Getting prior with the below method is equivalent to assuming no prior at all for the factors.
prior <- get_prior(data = newdata, bf(formula_full,family = bernoulli(link="logit")))
## define a prior on all population-level effects a once
prior$prior[1] <- "student_t(4,0,2.5)"
model1 <- brm(
data = newdata,
bf(formula_full, family = bernoulli(link="logit")),
prior = prior,
chains = 8, iter = 6000, warmup = 1000, #this makes total sample number 40,000
save_pars = save_pars(all = TRUE),
seed = 1015,
file = "twomain_SSxMask"
# chains = detectCores()-1
)
Thank you for your help in advance!
If I understand, from the interaction, you want the slope for the focus variable at different levels of of the moderator, right? These are called simple slopes in moderation analysis.
You can build the posterior distribution for them with the hypothesis function in brms
hypothesis(model1, "SS+ (SS:FT*10) == 0")
In this example, having SS as the focal predictor, and FT as the moderator. Testing the slope of SS when FT = 10
Hi Mauricio,
Thank you so much for your kind reply.
I tried your advice and I got the following result.
However, I am confused how to interpret this result. Can you advise something on the interpretation and if it possible, how I should write in a paper?
Also, the lower CI and the upper CI looks the same. Is it okay or normal?
Thank you so much!
Hi
Could you show the summary for the model and the syntax you ran?
I think I had an mistake on my code, it should just 1 =
hypothesis(model1, "SS+ (SS:FT*10) = 0")
Also, I used 10, just as an example value, but it might not be right for your data. Use meaningful values for the moderator variable
Hello Mauricio,
Thank you again for your reply and sorry for the late reply!
The following are the model summary and the syntax. In the below code, Facetype has three categorical factors and the Meandiff is a continuous variable.
Family: bernoulli
Links: mu = logit
Formula: p_negativeR ~ Facetype + Meandiff + Meandiff * Facetype + (1 | SID)
Data: newdata (Number of observations: 40080)
Draws: 8 chains, each with iter = 6000; warmup = 1000; thin = 1;
total post-warmup draws = 40000
Group-Level Effects:
~SID (Number of levels: 40)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 0.02 0.02 0.00 0.06 1.00 13793 19410
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept 0.13 0.02 0.09 0.17 1.00 33005 28091
FacetypehalfMcrowdmask 0.01 0.03 -0.04 0.06 1.00 37864 30544
Facetypemask -0.12 0.03 -0.17 -0.06 1.00 37563 30822
Meandiff 0.09 0.00 0.09 0.09 1.00 58186 30999
FacetypehalfMcrowdmask:Meandiff -0.03 0.00 -0.04 -0.03 1.00 57070 32366
Facetypemask:Meandiff -0.05 0.00 -0.05 -0.04 1.00 56761 33173
Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
After this, I did the following post-hoc analysis as you have recommended. “crowdmask” is one of the conditions of the Facetype variable.
> ht <- hypothesis(twomain_EmoxMask, "Meandiff+ (Facetypecrowdmask:Meandiff) > 0")
> ht
Hypothesis Tests for class b:
Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Post.Prob Star
1 (Meandiff+(Facety... > 0 0.05 0 0.05 0.06 Inf 1 *
---
'CI': 90%-CI for one-sided and 95%-CI for two-sided hypotheses.
'*': For one-sided hypotheses, the posterior probability exceeds 95%;
for two-sided hypotheses, the value tested against lies outside the 95%-CI.
Posterior probabilities of point hypotheses assume equal prior probabilities.
As I stated in the previous thread, I want to know how to interpret the estimate result and why the credible interval range is so small. If you by any chance have a paper that reported this kind of post-hoc analysis, can you share?
I really appreciate your help!
Sincerely,
Jieun
Jieun
Dont call this a post-hoc analysis. These are called simple slopes in moderation analysis, and have very specific use to explain “how” variables interact
The main issue with the 0 variabiloty is due to the slope of “Meandiff” , you can see in the summary of your model, that slope has SE=0. I would first recommend to look at this variable and this posterior, can help to plot it and see how narrow that posterior is.
Since your moderator is categorical, at the end, you would need 3 hypothesis
ht1 <- hypothesis(twomain_EmoxMask, "Meandiff > 0")
ht2 <- hypothesis(twomain_EmoxMask, "Meandiff + (Facetypecrowdmask:Meandiff)> 0")
ht3 <- hypothesis(twomain_EmoxMask, "Meandiff + (Facetypemask:Meandiff ) > 0")
Where ht1 is the slope of Meandiff when facetype is equal to the baseline category. So, ht2 is the slope of Meandiff when Facetype if equal to crowdmask. And ht3 is the slope of meandiff when facetype is equal to mask
For each of this, the estimate if the mean of the posterior, and the respective Crecibel Interval for each simple slope. Since you have an > sign, the Post.Prob colum indicate what proportion of the posterior is above 0 (in your case because > 0). So, in the case you shared, 100% of the posterior distribution is above 0
Hope this helps
1 Like
