I was wondering if somebody could briefly explain, in general, what the benefits are of using IS-LOO or K-fold CV as opposed to Bayes factors (or likelihood ratios I guess if carrying out a frequentist analysis)?

Hi,

First I gotta say I am not good at model selection, and there are better people that understand better the Bayes Factor and the bridge sampling algorithm (@bgoodri) or the LOO-CV and implementation with IS ( @jonah).

What I understand of Bayes Factor is that it looks if the model 's assumptions adjust well to the data, or well looks the best model. Even if this approach is a classical one and popular with some statisticians, have a lot of problems when the priors are not proper, and interpretate it is quite hard, if the bayes_factor is between some range then it doesnâ€™t decide well. Probably you have already read this article that explains it really well here.

On the other side, LOO-CV is more a non-parametric criterion, and its idea is to find the model with more predictive power. Thats why is more suitable for ML modelsâ€¦where all you need is a good forecast.

That is how I see it, hope it works, maybe others can give you an idea more accurately.

Asael.

I didnâ€™t implement the bridge sampling algorithm, but most of the cross-validation stuff and some of the comparisons to the Bayes factor approach are linked at

The main argument that proponents of cross-validation make is that it does not assume that one of the models that is to be estimated is correct.