My original motivation was the tau parameter in the 8 schools model. There, the central interval excludes zero–it has to!–but I wanted zero in the interval because zero is well supported by the data and model.
This is related to Sander Greenland’s idea of “compatability intervals” rather than “uncertainty intervals”: http://www.stat.columbia.edu/~gelman/research/published/uncertainty_intervals.pdf
In the 8 schools example, zero is fully compatible with the data and model, so I want to include it in the interval.
This is very much related to HPD reasoning. But I still think it’s hard to formalize the reasoning. It’s also hard to come up with a formal justification for a central 95% interval. Yes, the central interval is invariant to parameterization, but that’s not really a justification for the interval. It’s a pleasant property of the interval, once we’ve decided to use it, but it’s not a reason to use it in the first place.