Suppose I have a model with multiple categorical predictors, e.g.
response ~ ethnicity + religion
What’s the recommendation for setting up a weakly informative prior in this situation?
It wouldn’t make sense to use independent priors (e.g. Normal(0,1)) on all coefficients here, because then the prior predictive is asymmetric (the left out category has much less variance!)… So instead I’m usually inclined to just rewrite the model as a something like:
response ~ (1 | ethnicity) + (1 | religion)
…and set the σ prior to a constant. However:
(a) This feels inefficient, since it’s forcing stan to infer something I don’t care about – the μ for a population of unobserved ethnicities (/religions).
(b) The coefficients are no longer interpretable as I’d like them to be – for example, the posterior on the intercept (and the random effects) will continue to have uncertainty even in the infinite data limit.
It feels that it should just be possible to write the model without random effects at all, and using e.g. mean-centered predictors, but then just add some negative correlation into the prior to make the implied prior-predictive distribution the same for the left-out ethnicity (/religion) as for the others. Does anybody have any good tricks for this situation?