I asked how to conduct a Bayesian logistic regression in another post. I have been advised to use the brms R package.
I have conducted this analysis, but being a beginner in statistics and coding, I need to know if my analysis is right.
I have a data set with one binary outcome (“Object”). Participants had to choose between two objects. I want to check if this choice was not influenced by a continuous predictor (“PHQ”).
Only looking at the results of M1, I’d say that the evidence you have doesn’t really show a relationship between Objet and PHQ. Having said that, your credible intervals are pretty wide, and I wouldn’t be surprised to have only positive credible intervals if you include a 19th observation.
Both models had very similar LOO estimates, and the standard error for their formal contrast was larger than the difference itself. Thus, there’s little difference between the two models in the eyes of the LOO. Less formally, the PHQ seems to be about as good as nothing at all.
It doesn’t seem particularly informative. If you make a density plot of the posterior samples for the coefficient for PHQ, you will see that there is more probability for positive values, but the estimate for the parameter is compatible with negative, zero, and positive values. Since you are interested in a particular point hypothesis (that the coefficient is exactly zero), then you would hope to see the posterior distribution for the coefficient for PHQ to have statistics like the mean near to zero and a small standard error. How close to zero and how small a standard error is a question of how precise you need to be for your specific task. It seems that you only have 18 observations, so as @tomas.rossetti says, your posterior distribution for the coefficient for PHQ may very well change quite a bit with more observations. Since getting exactly zero won’t ever happen in the real world with real data given enough observations, then given an ever increasing amount of observations, your estimate for the coefficient will converge on some non-zero value with some decreasing SE, but will perhaps be for all practical purposes considered a non-existent effect. Say for example that you collect 999,982 more observations and re-run the model with more data and find now that the estimate is 0.002 (0.001-0.003). Maybe that’s small enough to be of no consequence, but it is measured more precisely. Trying to determine that something has no effect would seem difficult to me, because how far from exactly zero is good enough for ‘no effect’?
