Hey all -
So I’ve seen classic examples of say, if we have a binomoal likelihood, a coin flip, we want something like a N(.5,1) as a prior. (if we’re going for conjugacy, we want a beta binomal, but that’s not what we’re discussing).
So - I’m very fortunate to have come across Gabry et al’s “A visualization in …”.
I’m ok with posterior predictive checks. I’ve made blatantly obvious mistakes in the past, i.e. my “applied GPs in Stan” post. But I want to formalize the idea of a prior predictive check, especially when the likelihood is unknown. In this paper, I’m given visualizations with no code and it’s hard for me to formalize the idea.
For the coin flip example. I have a binomial RV. What’s P? I don’t know. Is my best guess still N(.5,1), or should I estimate from the data what my prior should be? What if the coin is bullshit, and it’s like a binomial with p=.0000000001, and I’ve guessed .5? I can do a simulation to show what my honest guesses do, but I want a more general answer with different likelihood functions. How much weight is the prior carrying?
Any papers/case studies/plain obvious examples I should look at?
FYI: posterior predictive checks - all about it! reject models regularly for unrealistic posterior predictive checks. The classical analogy is obvious extrapolation… same deal in machine learning and applied math…