This is a conceptual question and very likely stems from my own ignorance. Suppose you have a (toy) model like this:
Y \sim \text{Poisson}(\lambda),
\lambda \sim \text{Exponential}(1/\theta),
\theta \sim \text{Inverse-Gamma}(\alpha, \beta).
One can specify a program like
data{
int<lower=0> K;
int<lower=0> y[K];
real<lower=0> alpha;
real<lower=0> beta;
}
parameters{
real<lower=0> lambda;
real<lower=0> theta;
}
model{
target+= poisson_lpmf(y | lambda);
target+= exponential_lpdf(lambda | 1/theta);
target+= inv_gamma_lpdf(theta | alpha, beta);
}
to sample from this model and it works really well, as expected.
One can try to be smart and integrate out \theta, specifying the program
data{
int<lower=0> K;
int<lower=0> y[K];
real<lower=0> alpha;
real<lower=0> beta;
}
parameters{
real<lower=0> lambda;
}
model{
target+= poisson_lpmf(y | lambda);
target+= log(alpha) + alpha*log(beta)-(alpha+1)*log(lambda + beta);
}
which samples well, but achieves much lower ESS. Here’s the output for the full model and the marginalised one:
Inference for Stan model: toy_full.
4 chains, each with iter=2000; warmup=1000; thin=1;
post-warmup draws per chain=1000, total post-warmup draws=4000.
mean se_mean sd 2.5% 25% 50% 75% 97.5% n_eff Rhat
lambda 1.05 0.00 0.03 0.99 1.03 1.05 1.07 1.11 2925 1
theta 5.45 0.11 4.71 1.51 2.77 4.10 6.46 17.90 1995 1
lp__ -1329.94 0.03 1.05 -1332.78 -1330.34 -1329.61 -1329.19 -1328.92 1582 1
Inference for Stan model: toy_marginalised.
4 chains, each with iter=2000; warmup=1000; thin=1;
post-warmup draws per chain=1000, total post-warmup draws=4000.
mean se_mean sd 2.5% 25% 50% 75% 97.5% n_eff Rhat
lambda 1.05 0.00 0.03 0.99 1.03 1.05 1.07 1.12 1253 1
lp__ -1329.69 0.02 0.71 -1331.67 -1329.85 -1329.42 -1329.24 -1329.19 1541 1
I naively expected the marginalisation to improve sampling efficiency, but I guess there’s no guarantee that marginalising leads to better geometry, eh?
Tagging @betanalpha @avehtari @andrewgelman @bgoodri @Bob_Carpenter and @martinmodrak and apologising in advance if this is a stupid/trivial question.
