Interpreting results of hypothesis() when comparing one-sided and point hypotheses

Hi,
so there is a lot of conflicting statistical advice going around the Internet and real life, I hope I won’t be adding much to the cacophony. I understand if you are a bit overwhelmed.

What I think you are seeing is an instance of Bayes factors being confusing. I have to admit I never took time to get a good grip on Bayes factors because of such confusing properties (which I read about). I think a similar issue is discussed at [78a] If you think p-values are problematic, wait until you understand Bayes Factors - Data Colada (the tone of the post is a bit too agressive and snarky for my taste, but the explanation is nice) - Bayes Factor will integrate over your whole prior and are thus sensitive to your prior choice and to the way you choose your hypothesis.

Due to those confusing properties of Bayes factors most people here on the forums would probably not recommend using them broadly (just my guess - we didn’t have a vote or something).

So unless you have to use Bayes factors (your boss/reviewer/… wants it), I would just interpret the posterior distribution of the coefficient directly (mean, 50% interval, 95% interval at minimum). I.e. I would say something like “Provided that our model is a good approximation, the data cannot rule out negative effect (95% CI [-0.08, 0.41]) and are also consistent with negligible effect (posterior probability of $|b| < 0.X% is Y%), although strong positive effect is possible as well (posterior probability of $b > 0.Z is Q%)”.

If you totally need Bayes factors, then I’ll try to bring somebody with more background on BFs than I do.

I recently wrote an answer to a similar question which might also be of interest:

Hope that clarifies more than confuses and best of luck with your research!

1 Like