In what way are Gaussian priors "not robust" for logistic regression coefficients?

Ultimately, this is a semantic question about what “robust” means. Presumably, calling a normal prior “not robust” as a choice for a weakly informative prior reflects an idea that Normal priors tend to be too informative if what is desired is a weakly informative prior. They strongly suppress large coefficient values–perhaps more so than the modeler intends. They are not robust prior choices in the event that the true coefficient is quite large. You’re right; normal priors shrink large values more aggressively, which means that they might not be a robust choice of weakly informative prior model, which is generally intended not to shrink values too aggressively.

On the other hand, if the prior is intended to be informative, then it’s easy to imagine using the word “robust” to mean the exact opposite, as in normal priors are robust to atypical datasets and provide sufficient regularization of the coefficients whereas t priors do not. Whether or not that italicized statement is true depends entirely on which distribution (normal or t-with-finite-df) accurately captures the prior information that the modeler wishes to encode.

The recommendations are for the weakly informative case, and the point is that when we say “weakly informative”, we usually mean something with more mass in the tails than Gaussian.

4 Likes