Posterior estimates of rate and shape of gamma distribution are dependent

Modeling
14

TL;DR: Try reparameterizing with the mean and the [untransformed] shape parameter. Could someone please check this?

As a whimsical personal exercise, analogous to the normal I tried reparameterizing the gamma in terms of its mean and [untransformed] scale. It didn’t work, Fisher information off-diagonals were not 0.

On an even greater whim, I tried in terms of its mean and [untransformed] shape. And I think this transformation does orthogonalize the Fisher information. Given that I would imagine this has been tried before, I’d like others to check my work here.

Let the gamma distribution be parameterized by positive shape parameter \alpha and positive scale parameter \beta. The new parameterization is the mean \mu = \alpha\, \beta and untransformed shape \tau = \alpha. For x > 0, this new density is

\frac{\exp{(-x \tau / \mu)}\ x^{\tau - 1}} {(\mu / \tau)^\tau\ \Gamma(\tau) }

and thus its log density is

(\tau - 1) \log{(x)} + \tau (\log{(\tau)} - \log{(\mu)}) - \log{(\Gamma(\tau))} - x\, \tau / \mu,

or, rearranging for clarity,

\mathit{<univariate}\ \mathit{ stuff>} - \tau \log{(\mu)} - x\, \tau / \mu

The off-diagonal Hessian element of this is (x - \mu) / \mu^2. Clearly the (negative) expectation of this is zero.

It’s much harder to find errors when proofreading one’s own work versus proofreading others. Can someone confirm, or correct, this result? If this is confirmed, I would expect it to be a boon for both the OP and the community.

2 Likes