I was wondering if somebody could briefly explain, in general, what the benefits are of using IS-LOO or K-fold CV as opposed to Bayes factors (or likelihood ratios I guess if carrying out a frequentist analysis)?
1 Like
Hi,
First I gotta say I am not good at model selection, and there are better people that understand better the Bayes Factor and the bridge sampling algorithm (@bgoodri) or the LOO-CV and implementation with IS ( @jonah).
What I understand of Bayes Factor is that it looks if the model 's assumptions adjust well to the data, or well looks the best model. Even if this approach is a classical one and popular with some statisticians, have a lot of problems when the priors are not proper, and interpretate it is quite hard, if the bayes_factor is between some range then it doesn’t decide well. Probably you have already read this article that explains it really well here.
On the other side, LOO-CV is more a non-parametric criterion, and its idea is to find the model with more predictive power. Thats why is more suitable for ML models…where all you need is a good forecast.
That is how I see it, hope it works, maybe others can give you an idea more accurately.
Asael.
3 Likes
I didn’t implement the bridge sampling algorithm, but most of the cross-validation stuff and some of the comparisons to the Bayes factor approach are linked at
The main argument that proponents of cross-validation make is that it does not assume that one of the models that is to be estimated is correct.
2 Likes