I have data from 24 participants who completed each two conditions. I have computed the difference between the two conditions and I want to know whether that difference is negative (one-tailed t-test). Depending on the method I use, I get quite different results and I’m not sure which one is correct/ where I’ve made a mistake.
Methods I have used:
- hypothesis function on BRMS model
- bayes_factor function
- JASP coded as one-sample t.test
Here is a visualisation of the data from JASP:
model_priorBound= brm(CAPE.Pos_total_CDMminNDM ~ 0+Intercept,data=dfT,family=gaussian(),prior=set_prior("normal(0,1)",ub=0),iter=10000,warmup=1000,save_pars=save_pars(all=T))
model_noPriorBound=brm(CAPE.Pos_total_CDMminNDM ~ 0+Intercept,data=dfT,family=gaussian(),prior=set_prior("normal(0,1)"),iter=10000,warmup=1000,sample_prior = TRUE)
model_noIntercept= brm(CAPE.Pos_total_CDMminNDM ~ 0,data=dfT,family=gaussian(),iter=10000,warmup=1000,save_pars=save_pars(all=T))
The results are:
summary(summary(model_noPriorBound,prob=0.9)
Family: gaussian
Links: mu = identity; sigma = identity
Formula: CAPE.Pos_total_CDMminNDM ~ 0 + Intercept
Data: dfT (Number of observations: 24)
Draws: 4 chains, each with iter = 10000; warmup = 1000; thin = 1;
total post-warmup draws = 36000
Regression Coefficients:
Estimate Est.Error l-90% CI u-90% CI Rhat Bulk_ESS Tail_ESS
Intercept -0.59 0.50 -1.43 0.24 1.00 24878 21389
Further Distributional Parameters:
Estimate Est.Error l-90% CI u-90% CI Rhat Bulk_ESS Tail_ESS
sigma 2.84 0.42 2.24 3.61 1.00 24367 20714
→ 90% [one-tailed] credible interval includes 0: [-1.43;0.24]
hypothesis(model_noPriorBound,"Intercept<0")
Hypothesis Tests for class b:
Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Post.Prob Star
1 (Intercept) < 0 -0.59 0.5 -1.43 0.24 7.42 0.88
→ Bayes Factor: 7.42
bayes_factor(model_priorBound,model_noIntercept)
Estimated Bayes factor in favor of model_priorBound over model_noIntercept: 1.83863
And from JASP, I get a figure assessing the Bayes Factor sensitivity to the prior:
→ Bayes Factor: 0.92
When I set in JASP normal(0,1) instead of their default of cauchy(0,707), I get BF:0.938
0