Doubt regarding arviz's compare function

Modeling
1

Hello. I have a doubt regard arviz (az) compare function which. The output of

az.compare(model_dict)

contains elpd_loo and p_loo. I want to ask if elpd_loo is already penalized for the number of parameters, or do we need to do some manipulation with elpd_loo and p_loo (like elpd_loo - p_loo) to get the penalized score?

I ask this because I’m trying to compare two models where one of them has more parameters (one extra parameter). From what I have noticed so far, the model with more parameters has a higher elpd_loo. But this might be because my model hasn’t started overfitting yet.

Thanks for any insight on this!

2

It turns out there’s some really high-quality documentation of these terms for the Stan ecosystem in R. arviz uses them in the same way. So I would check out

elpd_loo is the Bayesian LOO estimate of the expected log pointwise predictive density, and so does not require any further penalization by the effective number of parameters.

1 Like