Creating null model for unequal variance model for Bayes factor calculation

Interfaces brms
1

Hello everyone,

for model comparison, I want to calculate Bayes factors. The models I ran with brms are ordinal models that allow for unequal variance by including the disc parameter. It seems to not be sensible to use the formula for unequal variances, because there are no different groups in the null model. Do I understand it correctly, that I can simply compare the models to the null model of the respective model with equal variances? Or do I have to fit the null model using the bf() and lf() formulas? If the latter is the case, could you maybe show me how this null model might look like for, e.g. the following model? Thank you!

fit3 <- brm(data = data,
            family = cumulative(probit),
            bf(Y ~ 1 + group) +
              lf(disc ~ 0 + group, cmc = FALSE),
            prior = c(prior(normal(0, 4), class = Intercept),
                      prior(normal(0, 2), class = b),
                      prior(normal(0, 1), class = b, dpar = disc))
2

Sorry for taking so long to reply, but I’ve used BF only once, and I generally dislike the approach since you have to be so careful with the priors. However, my hope is that @paul.buerkner, who has implemented this in brms, could help us here :)

3

Thank you for taking a look at this. Yes, I agree that BF should be used with caution. Which method would you recommend or prefer instead?

4

I would plot the distribution of the variances in each group and show how they differ visually. On top of this you could add a distribution where you only have an intercept model, i.e., \alpha to show how that looks like.