I just read the help for the loo_compare()
function in the brms package and it says
To compute the standard error of the difference in ELPD — which should not be expected to equal the difference of the standard errors — we use a paired estimate to take advantage of the fact that the same set of N data points was used to fit both models. These calculations should be most useful when N is large, because then non-normality of the distribution is not such an issue when estimating the uncertainty in these sums. These standard errors, for all their flaws, should give a better sense of uncertainty than what is obtained using the current standard approach of comparing differences of deviances to a Chi-squared distribution, a practice derived for Gaussian linear models or asymptotically, and which only applies to nested models in any case.
Does this mean you can compare non-nested models using the loo_compare()
function? And how far does this extend? Could you compare a linear model to a non-linear model for example? Or an exponential regression model to a gamma regression model.