I derived the PC prior (using Claude). It has a closed form, and a Fréchet distribution is a good simple approximation. I write \kappa = \alpha + \beta for the count parameter and \lambda for the rate of the PC prior, as in Dan’s blog post and in my negative-binomial notebook.
Base model. \theta \sim \mathrm{Beta}(\phi\kappa, (1-\phi)\kappa). The base model is \kappa = \infty: all \theta are equal to \phi, and there is no variation between groups.
Distance. The KL divergence from a beta distribution to a point mass is infinite. The precision of a Gaussian random effect has the same problem, and the same solution works: compute the KL divergence to \mathrm{Beta}(\phi\kappa_0, (1-\phi)\kappa_0) and let \kappa_0 \to \infty. Then
\mathrm{KL} = \kappa_0\, g(\kappa) - \tfrac{1}{2}\log\kappa_0 + O(1),
g(\kappa) = \phi\log\phi + (1-\phi)\log(1-\phi) + \psi(\kappa) - \phi\,\psi(\phi\kappa) - (1-\phi)\,\psi((1-\phi)\kappa).
The \log\kappa_0 term does not depend on \kappa, and the factor \kappa_0 goes into \lambda. Thus the distance is d(\kappa) = \sqrt{2g(\kappa)}. Here g(\kappa) is the expected KL divergence E_\theta[\mathrm{KL}(\mathrm{Bernoulli}(\phi) \,\|\, \mathrm{Bernoulli}(\theta))]. It decreases from \infty at \kappa = 0 to 0 at \kappa = \infty, thus the PC prior is proper on (0, \infty). The negative-binomial PC prior has the same structure: there the distance is \sqrt{2(\log s - \psi(s))} for the shape s of the gamma mixing distribution.
PC prior.
p(\kappa \mid \phi) = \lambda\, e^{-\lambda d(\kappa)}\, \frac{-g'(\kappa)}{d(\kappa)}, \qquad g'(\kappa) = \psi_1(\kappa) - \phi^2\psi_1(\phi\kappa) - (1-\phi)^2\psi_1((1-\phi)\kappa)
functions {
// PC prior for kappa in beta(phi * kappa, (1 - phi) * kappa), base model kappa = infinity
real pc_beta_kappa_lpdf(real kappa, real phi, real lambda) {
real g = phi * log(phi) + (1 - phi) * log1m(phi) + digamma(kappa)
- phi * digamma(phi * kappa) - (1 - phi) * digamma((1 - phi) * kappa);
real dg = trigamma(kappa) - square(phi) * trigamma(phi * kappa)
- square(1 - phi) * trigamma((1 - phi) * kappa);
real d = sqrt(2 * g);
return log(lambda) - lambda * d + log(-dg) - log(d);
}
}
Simple approximation. For large \kappa, 2g(\kappa) \approx 1/\kappa. For small \kappa, 2g(\kappa) \approx 2/\kappa. Thus d(\kappa)\sqrt{\kappa} is between 1 and \sqrt{2}:
| \phi \ \kappa |
0.1 |
1 |
10 |
100 |
1000 |
| 0.5 |
1.37 |
1.18 |
1.02 |
1.00 |
1.00 |
| 0.05 |
1.40 |
1.33 |
1.13 |
1.02 |
1.00 |
With d(\kappa) \approx \kappa^{-1/2}, the PC prior is an exponential prior on \kappa^{-1/2}, that is, a Fréchet distribution:
kappa ~ frechet(0.5, square(lambda));
This is the same form as the PC prior for the precision of a Gaussian random effect, because \kappa^{-1/2} is approximately \mathrm{sd}(\theta)/\sqrt{\phi(1-\phi)}. The approximation does not depend on \phi. It is accurate in the direction of the base model, and it puts more mass on small \kappa than the exact prior. Quantiles of \kappa for \lambda = 3:
| prior |
5% |
25% |
50% |
75% |
95% |
| exact, \phi = 0.5 |
1.3 |
5.1 |
19 |
109 |
3421 |
| exact, \phi = 0.05 |
1.7 |
6.4 |
21 |
112 |
3424 |
| Fréchet |
1.0 |
4.7 |
19 |
109 |
3421 |
Choosing \lambda. The PC prior gives P(\kappa < \kappa_L) = \exp(-\lambda\, d(\kappa_L)). With the approximation, \lambda = -\log(q)\sqrt{\kappa_L} gives P(\kappa < \kappa_L) = q. For example, \lambda = 3 gives P(\kappa < 1) \approx 0.05. Equivalently, \lambda \approx -\log(q)/U gives P(\mathrm{sd}(\theta) > U\sqrt{\phi(1-\phi)}) \approx q.
The priors in your post.
- BDA3 Section 5.3 uses p(\alpha, \beta) \propto (\alpha+\beta)^{-5/2}. This is a uniform prior on (\phi, \kappa^{-1/2}), thus p(\kappa) \propto \kappa^{-3/2}. The PC prior replaces the uniform prior on \kappa^{-1/2} with an exponential prior. It keeps the \kappa^{-3/2} tail, and it is proper.
- The User’s Guide prior p(\kappa) \propto \kappa^{-5/2} does not match BDA, because the Jacobian is missing. I made an issue: User's Guide: Beta count-parameter prior does not match the cited BDA prior · Issue #980 · stan-dev/docs · GitHub. On the scale u = \kappa^{-1/2}, that prior has density \propto u^2, which is zero at the base model.
- Gamma(4, 0.1) gives P(\kappa > 100) = 0.01, and Gamma(3, 0.04) gives P(\kappa > 100) = 0.24. On the scale u, the densities of both go to zero very fast as u \to 0. Thus both priors practically exclude models with almost no variation between groups: P(\kappa > 1000) < 10^{-14}. For comparison, the Fréchet prior with \lambda = 3 gives P(\kappa > 100) = 0.26 and P(\kappa > 1000) = 0.09. The gamma priors are good if you have prior information that the groups differ, but they do not shrink to the base model.
Dirichlet. The same derivation gives g(\kappa) = \sum_k \phi_k\log\phi_k + \psi(\kappa) - \sum_k \phi_k\psi(\phi_k\kappa). The approximation is an exponential prior on \sqrt{(K-1)/\kappa}, that is, kappa ~ frechet(0.5, (K - 1) * square(lambda)).
Notes.
- The exact prior depends on \phi, thus it is a conditional prior p(\kappa \mid \phi). The approximation does not depend on \phi.
- I defined the base model for \theta, as for the precision of a Gaussian random effect. The same prior applies to the precision parameter of a beta likelihood. For a beta-binomial model, you can also define the base model for the data (binomial). Then the KL divergence is finite, but it depends on the number of trials.
- I checked the KL limit, the normalization and the quantiles numerically in R, and the Stan function with a prior-only fit.