Hello,
Thank you for your response.
I have read this section, and many different topics in forums.
I think where I was confused is that when considering Evidence ratio, previous publication propose that: “Jeffreys recommends that odds greater than 3 be considered “some evidence,” odds greater than 10 be considered “strong evidence,” and odds greater than 30 be considered “very strong evidence” for one hypothesis over another.”
I thought that was the gold-standart. Which would mean, under such thresholds, both null and non-null hypothesis are considered valid.
But what we realized later is that in fact, the evidence ratio should much higher to be considered valid. As mentioned here.
A ER of 19 means a ratio of 0.95/0.05.
I know that interpretation is subjective obviously. But when considering the null effect hypothesis, usual method is to verify that 0 is not in the 95 or 90% confident interval.
As mentioned in the Rouder et al., the criterion of 3, 10 and 30 are more fitted for Bayes Factor, when other thresholds should be used for evidence ratio.
This was not an obvious feature of hypothesis() and there aren’t so many pages talking about this essential difference in reading the outputs of hypothesis either.
We probably still have a lot of frequentist biases in our way of validating hypothesis, but coming from a non mathematical background, it makes it difficult to adapt to hypothesis testing using hypothesis().
Thanks for your help