# Normalizing constant for embedded Laplace approximation

**URL:** <https://discourse.mc-stan.org/t/normalizing-constant-for-embedded-laplace-approximation/25573>\
**Category:** Algorithms\
**Created:** [December 5, 2021, 11:39pm UTC](https://discourse.mc-stan.org/t/normalizing-constant-for-embedded-laplace-approximation/25573 "2021-12-05T23:39:16Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![charlesm93](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/charlesm93/32/4634_2.png) [@charlesm93](https://discourse.mc-stan.org/u/charlesm93)\
**Post date:** [December 5, 2021, 11:39pm UTC](https://discourse.mc-stan.org/t/normalizing-constant-for-embedded-laplace-approximation/25573/1 "2021-12-05T23:39:16Z")

</div>

Do we need the normalizing constant of the likelihood when doing an embedded Laplace approximation? It seems that we can drop the constant and make computations a bit more efficient.

Given a model

\phi \sim \pi(\phi) \\ \eta \sim \pi(\eta) \\ \theta \mid \phi \sim \text{Normal}(0, K(\phi)) \\ y \mid \theta, \eta \sim \pi(y \mid \theta, \eta)

and introducing the negative Hessian, W = - \nabla^2\_\theta \log \pi(y \mid \theta^\*, \eta), the approximate marginal likelihood is

\log \pi\_\mathcal{G} (y \mid \phi, \eta) = - \frac{1}{2} \theta^{\*T} K^{-1} \theta^\* + \log \pi(y \mid \theta, \eta) - \frac{1}{2} \log |K||K^{-1} + W|. 

When drawing approximate samples from the marginal posterior \pi(\phi, \eta \mid y), we don’t need the normalizing constant in \log \pi(y \mid \theta^\*, \eta). Our current unit tests benchmark against GPStuff, which as far as I can tell computes the normalizing constant. Is there a reason for this?

**Edit** : making corrections to the equations, following Aki’s suggestion.

@avehtari

---

<div class="post-metadata">

**Author:** ![avehtari](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/avehtari/32/5935_2.png) [@avehtari](https://discourse.mc-stan.org/u/avehtari)\
**Post date:** [December 6, 2021, 7:04pm UTC](https://discourse.mc-stan.org/t/normalizing-constant-for-embedded-laplace-approximation/25573/2 "2021-12-06T19:04:19Z")

</div>

> [@charlesm93](#):
>
> the approximate marginal likelihood is
> 
> \log \pi\_\mathcal{G} (y \mid \phi, \eta) = - \frac{1}{2} \theta^{\*T} K^{-1} \theta^\* + \log \pi(y \mid \theta, \eta) - \frac{1}{2} \log |K||K^{-1} + W|.

I think \log \pi(y \mid \theta, \eta) should be \log \pi(y \mid \theta^\*, \eta)

> [@charlesm93](#):
>
> When drawing approximate samples from the marginal posterior \pi(\phi \mid y)

I think \pi(\phi \mid y) should be \pi(\phi, \eta \mid y)

> [@charlesm93](#):
>
> we don’t need the normalizing constant in \log \pi(y \mid \theta, \eta)

I think \log \pi(y \mid \theta, \eta) should be \log \pi(y \mid \theta^\*, \eta). As \theta^\* is fixed, the normalization term with respect to \theta^\* is constant and not needed for MCMC, but in most cases the computational cost is negligible compared to matrix and Laplace operations, and it was easier to make comparisons to full MCMC when not dropping those terms. For models with \eta as parameter, the normalization term depends on \eta and needs to be included.

---

<div class="post-metadata">

**Author:** ![charlesm93](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/charlesm93/32/4634_2.png) [@charlesm93](https://discourse.mc-stan.org/u/charlesm93)\
**Post date:** [December 6, 2021, 7:57pm UTC](https://discourse.mc-stan.org/t/normalizing-constant-for-embedded-laplace-approximation/25573/3 "2021-12-06T19:57:26Z")

</div>

Ok, let’s plan to keep the normalizing constant for now. I’ll do some checks to examine performance differences.