# Estimated p\_loo \> p

**URL:** <https://discourse.mc-stan.org/t/estimated-p-loo-p/8671>\
**Category:** Modeling\
**Tags:** loo\
**Created:** [April 30, 2019, 9:34pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671 "2019-04-30T21:34:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nlpacestan](https://avatars.discourse-cdn.com/v4/letter/n/71e660/32.png) [@nlpacestan](https://discourse.mc-stan.org/u/nlpacestan)\
**Post date:** [April 30, 2019, 9:34pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671/1 "2019-04-30T21:34:13Z")

</div>

rstanarm 2.18.2  
loo.1.0  
Windows

Using rstanarm I obtained a stan\_lm model with 37 predictors and 3 auxiliary parameters, N = 1104358. The R2 prior was 0.5.

The loo object returns:

Computed from 16000 by 1104358 log-likelihood matrix

```
       Estimate SE

```

## elpd\_loo -1090026.8 995.6 p\_loo 41.6 0.1 looic 2180053.6 1991.1

Monte Carlo SE of elpd\_loo is 0.1.

All Pareto k estimates are good (k \< 0.5).

The largest k value was 0.136.

The loo-glossary states that for p\_loo \> p, the model has weak predictive capability and may indicate model mispecification.

How concerned should I be?

Are there other relevant loo model checks?

Nathan

---

<div class="post-metadata">

**Author:** ![bgoodri](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bgoodri/32/4451_2.png) [@bgoodri](https://discourse.mc-stan.org/u/bgoodri)\
**Post date:** [April 30, 2019, 9:46pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671/2 "2019-04-30T21:46:38Z")

</div>

I would upgrade to **loo** 2.x. You can do all the regular posterior predictive checks / plots, but it is going to be unwieldy and memory-intensive with over a million observations.

---

<div class="post-metadata">

**Author:** ![nlpacestan](https://avatars.discourse-cdn.com/v4/letter/n/71e660/32.png) [@nlpacestan](https://discourse.mc-stan.org/u/nlpacestan)\
**Post date:** [April 30, 2019, 10:05pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671/3 "2019-04-30T22:05:20Z")

</div>

I incorrectly entered the loo version.

I am using 2.1.0.

Nathan

---

<div class="post-metadata">

**Author:** ![avehtari](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/avehtari/32/5935_2.png) [@avehtari](https://discourse.mc-stan.org/u/avehtari)\
**Post date:** [May 1, 2019, 7:32pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671/4 "2019-05-01T19:32:48Z")

</div>

> [@nlpacestan](#):
>
> Using rstanarm I obtained a stan\_lm model with 37 predictors and 3 auxiliary parameters, N = 1104358.

With that model and N very much bigger than the number of parameters, you don’t need loo for model comparison.

> [@nlpacestan](#):
>
> The loo-glossary states that for p\_loo \> p, the model has weak predictive capability and may indicate model mispecification.

I should probably chance that to p\_loo \> p +1, as p\_loo can in this kind of cases be by small amount larger than p. I guess that in addition of 37 predictors and 3 auxiliary parameters there is also intercept term, so that p=41.

> [@nlpacestan](#):
>
> How concerned should I be?

No need for concern.

btw. there will be soon a new loo version which will compute loo for this at 1000 times faster using ideas presented in [Bayesian leave-one-out cross-validation for large data](https://arxiv.org/abs/1904.10679)

---

<div class="post-metadata">

**Author:** ![nlpacestan](https://avatars.discourse-cdn.com/v4/letter/n/71e660/32.png) [@nlpacestan](https://discourse.mc-stan.org/u/nlpacestan)\
**Post date:** [May 1, 2019, 10:37pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671/5 "2019-05-01T22:37:30Z")

</div>

Aki, I appreciate your comments. I have been reading (heavy going for me) your recent papers on importance sampling. I have also read Gelman, Carlinf (2013) on the derivation of effective parameter estimates. Still don’t get why the lppd informs one about the parameter count. I’ll keep reading.

The 37 predictors included the intercept, but I have stopped being concerned.

I will read your new paper about loo for large data. I can run the current model in 20-30 minutes. Running loo takes 5+ longer. There is an eta for the next loo package update?

---

<div class="post-metadata">

**Author:** ![avehtari](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/avehtari/32/5935_2.png) [@avehtari](https://discourse.mc-stan.org/u/avehtari)\
**Post date:** [May 2, 2019, 7:58pm UTC](https://discourse.mc-stan.org/t/estimated-p-loo-p/8671/6 "2019-05-02T19:58:05Z")

</div>

> [@nlpacestan](#):
>
> There is an eta for the next loo package update?

Not certain about CRAN relase, but at least github version with couple exciting new features hopefully before the end of May (but I seem to be often a bit optimistic).