I have 2 models fitted on few datasets that I would like to compare. They are nested so that model 2 is a special case of model 3.
I expect that in some datasets, the added parameters in model 3 will be relevant but not in all of them, and I am unsure on how to test this.
Should I simply use PSIS-LOO? In which case, I should only consider differences in elpd to be significant when they are above 5 SE, is that correct?
In this case, given that my models are nested, can model 2 outperform model 3? Or should I interpret a small difference in elpd relative to the SE as there being no advantage of adding parameters to my model?
Would computing stacking weights make sense here? I’m still struggling a bit to understand how they work.
Should I rather only fit my full model and look at the estimated parameters? Or do something else entirely maybe?
Thank you for the help