# How can I acquire the log likelihood from "loo"?

**URL:** <https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048>\
**Category:** Modeling\
**Tags:** loo\
**Created:** [June 16, 2021, 2:29pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048 "2021-06-16T14:29:49Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![hawe66](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/hawe66/32/11902_2.png) [@hawe66](https://discourse.mc-stan.org/u/hawe66)\
**Post date:** [June 16, 2021, 2:29pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/1 "2021-06-16T14:29:49Z")

</div>

I generated log\_lik at the generated quantities block,  
and I extracted loo by

```
LL_2 <- extract_log_lik(output, parameter_name = "log_lik", merge_chains = TRUE)
loo_2 <- loo(LL_2, save_psis = TRUE, cores = 4)
waic_2 <- waic(LL_2, save_psis = TRUE, cores = 4)

```

Anyway, I’m curious if elpd\_loo and waic\_loo are the same as the conventional log-likelihood…  
They produced the same value but not the value that I expected (so close to zero).  
Are they the real log-likelihood or should I have to multiply somehow?

---

<div class="post-metadata">

**Author:** ![jonah](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jonah/32/200_2.png) [@jonah](https://discourse.mc-stan.org/u/jonah)\
**Post date:** [June 16, 2021, 10:03pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/2 "2021-06-16T22:03:48Z")

</div>

> [@hawe66](#):
>
> Are they the real log-likelihood or should I have to multiply somehow?

Can you clarify a bit more? The real log-likelihood values are the ones you computed in your Stan program and extracted, but I’m guessing I’m not understanding what you’re actually asking.

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [June 17, 2021, 3:55am UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/3 "2021-06-17T03:55:08Z")

</div>

As datasets get large, likelihoods get really small. For a really simple example, consider the Bernoulli likelihood for p = 0.5. With one data point, that likelihood will be 0.5. With 2 data points, the likelihood will be .25. With N data points, the likelihood will be 2^{-N}. By the time you get to even modest-size datasets, the likelihood gets vanishingly small. In fact, this is part of the motivation for working with log-likelihoods instead of likelihoods; the likelihood itself tends to underflow to zero. You can expect likelihoods close to zero for models with more than a handful of data points.

---

<div class="post-metadata">

**Author:** ![hawe66](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/hawe66/32/11902_2.png) [@hawe66](https://discourse.mc-stan.org/u/hawe66)\
**Post date:** [June 17, 2021, 2:10pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/4 "2021-06-17T14:10:01Z")

</div>

Sorry for unclear explanation.  
I declared vector[25] log\_lik and I got [1:6000,1:25] size of log\_lik after MCMC sampling.  
And I extracted log-likelihood by loo package and this code.

```
output = stan("steak.stan", data = dataList, 
              control = list(adapt_delta = 0.99, max_treedepth = 15),
              pars = c("w", "pi", "lambda", "alpha", "log_lik"), init_r=5,
              iter = 6000, warmup=1000, thin=3, chains=4, cores=4)

LL_2 <- extract_log_lik(output, parameter_name = "log_lik", merge_chains = TRUE)

loo_2 <- loo(LL_2, save_psis = TRUE, cores = 4)

print(loo_2)

```

But my question was on the result of print(loo\_2)

```
Computed from 6668 by 25 log-likelihood matrix

         Estimate SE
elpd_loo -10.5 1.4
p_loo 0.2 0.0
looic 21.0 2.9
------
Monte Carlo SE of elpd_loo is 0.0.

```

The expected log likelihood is above 1000, based on the prior research.  
But estimated elpd\_loo was 10.5. Is this appropriate value, or did I do something wrong?

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [June 17, 2021, 2:18pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/5 "2021-06-17T14:18:22Z")

</div>

`elpd_loo` is not the same thing as the fitted log likelihood. See here [LOO package glossary — loo-glossary • loo](https://mc-stan.org/loo/reference/loo-glossary.html)  
But also, it’s very difficult for me to imagine a dataset that yields a likelihood of e^{1000}. I’m not sure what you mean by “the prior research”, but are you sure this is a reliable number?

Edit: I guess this is possible if the likelihood functions are really strongly peaked.

---

<div class="post-metadata">

**Author:** ![hawe66](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/hawe66/32/11902_2.png) [@hawe66](https://discourse.mc-stan.org/u/hawe66)\
**Post date:** [June 17, 2021, 2:29pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/6 "2021-06-17T14:29:57Z")

</div>

Thanks for the kind explanation!

Actually I was trying to analyze the open dataset, and the BIC score from the original article was around 6400. This is the summation of the BIC score of 25 participants’ data, and each of them did 180 trials. (So that being the summation of 25 \* 180 = 4500 log-likelihoods.)

Anyway, if `elpd_loo` doesn’t actually mean the fitted log-likelihood, then how can I compute the fitted log-likelihood with loo package?

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [June 17, 2021, 2:31pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/7 "2021-06-17T14:31:53Z")

</div>

The BIC uses the _negative_ log likelihood.

---

<div class="post-metadata">

**Author:** ![hawe66](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/hawe66/32/11902_2.png) [@hawe66](https://discourse.mc-stan.org/u/hawe66)\
**Post date:** [June 17, 2021, 2:33pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/8 "2021-06-17T14:33:43Z")

</div>

Oh I just didn’t mention the sign. Their BIC score was -6149.

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [June 17, 2021, 2:43pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/9 "2021-06-17T14:43:10Z")

</div>

> [@hawe66](#):
>
> Anyway, if `elpd_loo` doesn’t actually mean the fitted log-likelihood, then how can I compute the fitted log-likelihood with loo package?

Sum over the rows of `loo::extract_log_lik(..., merge_chains=TRUE)`. This assumes of course that the log likelihood is correctly computed in your stan model itself.

---

<div class="post-metadata">

**Author:** ![jsocolar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jsocolar/32/2486_2.png) [@jsocolar](https://discourse.mc-stan.org/u/jsocolar)\
**Post date:** [June 17, 2021, 2:51pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/10 "2021-06-17T14:51:48Z")

</div>

Note also that the BIC uses the maximum value of the likelihood function, whereas when fitting the model you’ll get the “typical” value of the likelihood function. These can be rather different.

---

<div class="post-metadata">

**Author:** ![hawe66](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/hawe66/32/11902_2.png) [@hawe66](https://discourse.mc-stan.org/u/hawe66)\
**Post date:** [June 17, 2021, 3:00pm UTC](https://discourse.mc-stan.org/t/how-can-i-acquire-the-log-likelihood-from-loo/23048/11 "2021-06-17T15:00:39Z")

</div>

Thanks a lot! I learned a lot from your replies!!! Have a really really nice day :)