Bayesian models - choosing between Means, Medians, MAD, MPE

Hi everyone,

I’m currently running mixed models in brms, and reading lots of papers that have done similar. I’m getting a little confused now about my approach to identifying and summarising estimates that have meaningful effects.

I’ve seen lots of people using whether the posterior distributions cross 0 or not (95 or 89 % credible intervals)…based on the means or medians.

I’ve also seen people summarising the posterior distributions with medians and then using maximum probability of effect (MPE) estimates to make decisions on what is meaningful.

In another paper, the authors used the posterior median, and the median absolute deviation (MAD) to assess the strength and uncertainty of estimated effects.

My question is…how does one make a decision on whether to use the posterior mean or median, and whether or not to use just those, or to also explore MPEs/MADs?

Any help would be much appreciated.

Reasonable people disagree about this, or just have their own personal preferences/standards without necessarily thinking others are better or worse. Essentially, this question is about how to summarise your posterior, and the idea is to get a) a point estimate indicating some kind of reasonable estimate of the posterior, and b) another number or set of numbers indicating the spread of the posterior around that point estimate, to indicate uncertainty. There are just a lot of ways of doing this!

There are some advantages to a median, in that the value indicated by the median estimate does not change with a transformation of the posterior (e.g., a log transform, it would indicate the same value, whereas the mean might change). The same applies to intervals based on a quantile from the posterior (an equal tailed interval, which is what brms provides by default, is the 2.5% and 97.5% of the posterior, and the values indicated by these also do not change when the posterior is transformed).

However, one could also use the HDI - the highest density interval. A 95% HDI indicates the 95% most likely values in the posterior, whereas the 95% ETI just indicates the middle 95% of values - not necessarily the most likely.

Things like the MAD also just indicate the spread of the value around the median.

As you can see, this can be quite an annoying thing to decide which to use. It is not the case the one is necessarily better than the other. Personally, I am often happy to use the median, mean, or even mode (most likely part of the posterior distribution) as a point estimate, and some kind of HDI as the indicator of uncertainty. However, what I would suggest is seeing if your interpretations of what is going on actually change dramatically depending on how you summarise the posterior. If your posterior has a heavy skew, then the different ways of summarising it can make a difference. Plot a density plot of the posterior and see the shape, and how the summary points line up with one another. Often, you will be relieved to see that it doesn’t make a difference. If the shape of the posterior is very skewed or has a weird shape, this might be a case where you would want to literally show a plot of the posterior so that people can see what is going on, and not be mislead by a potentially arbitrary choice of summary.

Finally, regarding using whatever %age of the posterior, this depends on how stringent you want to be about the estimate. A 99% interval contains more of the posterior than a 95%, and more again than an 89%. You can think of it like your acceptable error rate - for a 99% HDI, you expect that it is 99% likely that the true value of the underlying parameter is within that range. So the preference here depends on how conservative you want to be.

4 Likes

Thank you very much @JimBob, that is really helpful!
I also just checked my posterior density plots, and they are quite symmetrical.

The HDI sounds very useful - presumably it is still reasonable to use the HDI with the mean estimates?

Also, is there a reason (other than higher confidence) why the HDI interval is quite a lot higher (99%) than typically used intervals e.g., 89%, 95%?

The reason to choose different intervals is often to avoid ‘false positives’ or appearing overly certain about where the parameter lies. Notably, ‘89%’ was suggested almost as a joke by Richard McElreath ‘because it is prime’ and is specifically meant to be arbitrary and cause you to think of what %age you actually want - not to be taken as a standard with any meaning. 95% interval essentially comes as a parallel to .05 p value or 95% confidence interval in frequentist statistics. Personally I think 89% might not be conservative enough, but the choice is up to you and depends on your goal.

You can combine the HDI with the mean, median or whatever else you like, yes.

1 Like