Hello, I have some data from a pilot study and plan to analyze the (new) data of the main experiment. My idea was to use the posteriors of the pilot analyses as priors for the Bayesian analyses of the main experiment. However, I have a very basic question: Is it advisable to inform the prior for my hypothesized effect based on the hypothesis? More specifically, the pilot data provided an unexpected effect (it went into the “wrong”/other direction) and I am now asking myself whether to adjust the prior into the expected direction as this is my expectation or if this is inherently wrong and problematically biases the data into the direction of my hypothesis. I would be very thankful for any help
You should use the posterior of the pilot data as the prior for the new data assuming that it is the same experiment and model. Adjusting the prior in the direction of the hypothesis would unduly stack evidence in its favor that would lead to an illusory strength of evidence that does not exist.
1 Like
I don’t think this is straightforward. Yes, basing the prior for the main experiment on the pilot results could be a good idea IF the study protocols and population from which you’re drawing your samples are all the same. If you’ve updated either of those, I’d use caution.
I don’t think it’s unreasonable to base your prior on your substantive hypothesis. But doing so puts you in a position where you are making a hypothesis despite the pilot, and I wouldn’t be surprised if folks in your field found that a hard pill to swallow.
So perhaps ask yourself: If I was a skeptical reviewer, which method would I think was the most rigorous?
2 Likes