# Yet another lp\_\_ question

**URL:** <https://discourse.mc-stan.org/t/yet-another-lp-question/3827>\
**Category:** General\
**Created:** [April 10, 2018, 3:25pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827 "2018-04-10T15:25:48Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![moen](https://avatars.discourse-cdn.com/v4/letter/m/c5a1d2/32.png) [@moen](https://discourse.mc-stan.org/u/moen)\
**Post date:** [April 10, 2018, 3:25pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/1 "2018-04-10T15:25:48Z")

</div>

Dear Stan community

First of all thanks for the great job! Stan really nails it!

My question regards the use of lp\_\_ for model comparison.  
If I got it well, if I have written my model with the syntax `x ~ normal(m,s)` instead of `target += lpdf_normal(x | m,s)` then my lp\_\_ is the log-joint minus a constant, is that correct?

I want to compare an approximate inference algorithm - some complex stuff that I did not code with Stan - with NUTS, so I would like to know the value of the constant for model comparison.  
Is there a way to get that back without re-running everything?

I am using cmdstan on a distant cluster (I have to run lots of simulations with lots of data) and I would like to keep the runs that I have done - if possible.

Thanks for your precious help

---

<div class="post-metadata">

**Author:** ![bgoodri](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bgoodri/32/4451_2.png) [@bgoodri](https://discourse.mc-stan.org/u/bgoodri)\
**Post date:** [April 10, 2018, 3:29pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/2 "2018-04-10T15:29:27Z")

</div>

> [@moen](#):
>
> my lp\_\_ is the log-joint minus a constant

Only if your parameters block contains unrestricted parameters exclusively. Otherwise, the `lp__` column of the output is going to have Jacobian terms.

> [@moen](#):
>
> Is there a way to get that back without re-running everything?

No. Although a more principled way of evaluating approximate inference algorithms is dropping any day now.

---

<div class="post-metadata">

**Author:** ![moen](https://avatars.discourse-cdn.com/v4/letter/m/c5a1d2/32.png) [@moen](https://discourse.mc-stan.org/u/moen)\
**Post date:** [April 10, 2018, 3:31pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/3 "2018-04-10T15:31:19Z")

</div>

Ok great. thanks for the fast answer. If I change the syntax to `target +=`, will that solve my problem?

---

<div class="post-metadata">

**Author:** ![bgoodri](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bgoodri/32/4451_2.png) [@bgoodri](https://discourse.mc-stan.org/u/bgoodri)\
**Post date:** [April 10, 2018, 4:01pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/4 "2018-04-10T16:01:47Z")

</div>

That will mean that the results match what you would calculate outside of Stan (if you adjusted for the change-of-variables), but I don’t think `lp __` is much use for comparing models / algorithms. ELPD is good, but that just uses the likelihood and only for one observation at a time in the case of the PSIS correction. The marginal likelihood has its proponents, but that does not correspond to `lp__ ` either.

---

<div class="post-metadata">

**Author:** ![moen](https://avatars.discourse-cdn.com/v4/letter/m/c5a1d2/32.png) [@moen](https://discourse.mc-stan.org/u/moen)\
**Post date:** [April 10, 2018, 4:18pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/5 "2018-04-10T16:18:08Z")

</div>

True! I do not intend to use lp\_\_ to select the best model, but rather to measure a KL divergence between an VB ELBO and the true posterior probability. Just to know how bad (or good) I’m doing with VB. If I’m not wrong, with an accurate lp\_\_, I can estimate the log-marginal ‘log p(y)’ as  
logsumexp(lp\_\_)-log(N)  
right?  
Would you advice some other way to do that?

---

<div class="post-metadata">

**Author:** ![bgoodri](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bgoodri/32/4451_2.png) [@bgoodri](https://discourse.mc-stan.org/u/bgoodri)\
**Post date:** [April 10, 2018, 4:26pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/6 "2018-04-10T16:26:15Z")

</div>

I would wait a couple of days.

---

<div class="post-metadata">

**Author:** ![avehtari](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/avehtari/32/5935_2.png) [@avehtari](https://discourse.mc-stan.org/u/avehtari)\
**Post date:** [April 10, 2018, 4:51pm UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/7 "2018-04-10T16:51:10Z")

</div>

> [@moen](#):
>
> Just to know how bad (or good) I’m doing with VB.

Yes, but Did It Work?: Evaluating Variational Inference. [[1802.02538] Yes, but Did It Work?: Evaluating Variational Inference](https://arxiv.org/abs/1802.02538)

---

<div class="post-metadata">

**Author:** ![moen](https://avatars.discourse-cdn.com/v4/letter/m/c5a1d2/32.png) [@moen](https://discourse.mc-stan.org/u/moen)\
**Post date:** [April 11, 2018, 5:00am UTC](https://discourse.mc-stan.org/t/yet-another-lp-question/3827/8 "2018-04-11T05:00:16Z")

</div>

Fantastic! Thanks a lot!