AIC, WAIC, LOO-CV for the survival joint model in Stan

Most of the packages in the Stan ecosystem (e.g. the rstanarm::stan_jm implementation of the survival-longitudinal joint model) are integrated with the loo package that provides both PSIS-LOO (recommended) and WAIC (discouraged) computations.

To have loo work with a custom model, you need to compute separate log likelihood for each unit you are planning to leave out (this would usually be subjects, but don’t have to be). An example on how to do this is at Writing Stan programs for use with the loo package • loo

I am not sure I understand what you mean here. Note that if some value needs to be reused in both model and generated quantities block, you can compute it in transformed parameters - the transformed parameters are available in both subsequent blocks.

You don’t need cross validation for Bayesian analysis. It is just a step that many people found helpful There are other criteria you can use to compare models and/or select hypothesis (my current thinking on the topic is at Hypothesis testing, model selection, model comparison - some thoughts. Or you may not need to compare/select at all!

Regarding the “different priors” part - cross validation only tries to estimate which of the models (including their priors) will do best in a given prediction task. So the priors matter only to the extent they influence the posterior predictions of the models. The Cross Validation FAQ has a bit more details on when and how you might find CV useful.

Best of luck with your model!

3 Likes