I’m having some issues with computing Bayes factors for two competing models. Both models seem to fit fine and I can compute information criteria and extract parameter estimates. However, when I use bridge sampling to calculate Bayes factors, one model appears strongly preferred (estimated Bayes factor is extraordinarily, unrealistically high). This seems inaccurate, as the two models are very similar in WAIC and the extra parameter in the more complex model has very little effect.
I set the number of samples to be much higher, since I’m using bridge sampling (around 18000). The bridge sampler isn’t reaching its iteration limit (took around 64 iterations for the more complex model).
I’m just wondering what might be causing this mismatch. I’ve included the more complex model summary below, all priors are relatively weak, regularising priors. The simpler model is the same, only without the bPos parameter and bPos * Pos term in the model.
Any help is much appreciated!
parameters{
// Group-level hyperparameters
real mu_bCons;
real mu_bSymm;
real mu_bAcc;
real mu_bPos;
real<lower=0> sigma_bCons; // sd of group-level distribution
real<lower=0> sigma_bSymm; // sd of group-level distribution
real<lower=0> sigma_bAcc; // sd of group-level distribution
real<lower=0> sigma_bPos; // sd of group-level distribution
// Subject-level parameters (raw)
vector[N] bCons; // Constant coeff
vector[N] bSymm; // Asymm coeff
vector[N] bAcc; // Accuracy coeff
vector[N] bPos; // Positivity coeff
vector<lower=0>[N] sigma_ID; // Vector of std of subject-level updates
}
model {
// Group-level priors
mu_bCons ~ normal(0, 1);
mu_bSymm ~ normal(0, 1);
mu_bAcc ~ normal(0, 1);
mu_bPos ~ normal(0, 1);
sigma_bCons ~ gamma(1, 0.5);
sigma_bSymm ~ gamma(1, 0.5);
sigma_bAcc ~ gamma(1, 0.5);
sigma_bPos ~ gamma(1, 0.5);
// Subject_level priors
bCons ~ normal(mu_bCons, sigma_bCons);
bSymm ~ normal(mu_bSymm, sigma_bSymm);
bAcc ~ normal(mu_bAcc, sigma_bAcc);
bPos ~ normal(mu_bPos, sigma_bPos);
// Vectorised likelihood
UpdateS ~ normal((bCons[S] + bAcc[S] .* Acc + bPos[S] .* Pos) .* (1 + (bSymm[S] .* Dir)), sigma_ID[S]);
}