Hi!
I’m fitting models using brms package. I know that WAIC and LOO can be computed by it. Are these fit indices based on conditional likelihood or marginal likelihood? How can I compute marginal likelihood in stan? And use it to obtain marginal WAIC and LOO?
Thank you so much in advance!
General answer to getting the WAIC/LOO from brms:
model_fit <- add_criterion(model_fit, criterion = "loo")
As far as extracting the marginal likelihood, I believe that this needs to be done via the bridgesampling package. Here’s the page on it from the brms package: Log Marginal Likelihood via Bridge Sampling — bridge_sampler.brmsfit • brms. This requires a much larger number of sampling iterations than usually recommended/needed by Stan, so just an FYI since you seemingly are most interested in this marginal value
If the question is referring to Bayesian Comparison of Latent Variable Models: Conditional Versus Marginal Likelihoods | SpringerLink, then see a case study Roaches cross-validation demo, where in the case of LOO the marginalization is called integrated LOO. The same approach can be used when each group has more than one observation. The demo shows how to integrate out one group specific parameter, but there is in some thread in discourse also an example by someone else showing how to do nested integration for two parameters.