# Non centered parameterization on variance parameter

**URL:** <https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584>\
**Category:** Modeling\
**Created:** [May 17, 2017, 7:38pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584 "2017-05-17T19:38:08Z")\
**Posts on this page:** 12\
**Page:** 2

<div class="post-metadata">

**Author:** ![sakrejda](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sakrejda/32/845_2.png) [@sakrejda](https://discourse.mc-stan.org/u/sakrejda)\
**Post date:** [May 23, 2017, 7:41pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/21 "2017-05-23T19:41:19Z")

</div>

Yeah, even with multiple chains the metrics seem not to do a great job. I  
wonder if energy does (haven’t looked) but it is a pretty obvious pattern  
on the trace plots and I would always check for it if I saw heavy tails

---

<div class="post-metadata">

**Author:** ![sakrejda](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sakrejda/32/845_2.png) [@sakrejda](https://discourse.mc-stan.org/u/sakrejda)\
**Post date:** [May 23, 2017, 7:52pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/22 "2017-05-23T19:52:32Z")

</div>

isn’t it the derivative of the inverse? I always have to check to remember…

---

<div class="post-metadata">

**Author:** ![betanalpha](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/betanalpha/32/4_2.png) [@betanalpha](https://discourse.mc-stan.org/u/betanalpha)\
**Post date:** [May 23, 2017, 8:05pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/23 "2017-05-23T20:05:02Z")

</div>

The energy diagnostic will identify heavy tails but as far as I can tell dynamic HMC handles them just fine. Stan can nail down the 1% and 99% quantiles of a 100-dimensional Cauchy in just a few minutes.

---

<div class="post-metadata">

**Author:** ![sakrejda](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sakrejda/32/845_2.png) [@sakrejda](https://discourse.mc-stan.org/u/sakrejda)\
**Post date:** [May 23, 2017, 8:11pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/24 "2017-05-23T20:11:29Z")

</div>

Huh. What do you think is happening in this model then?

---

<div class="post-metadata">

**Author:** ![aaronjg](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/aaronjg/32/3768_2.png) [@aaronjg](https://discourse.mc-stan.org/u/aaronjg)\
**Post date:** [May 25, 2017, 1:43am UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/25 "2017-05-25T01:43:22Z")

</div>

As per Michael’s comment, the jacobian is not needed. Even though the likelihood functions appear different, it seems that you are actually calculating

 ![](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/1X/f2436cadcca82da61b41593acbcd0e1e2ce1cada.png)

and so even though p(hp\_alt) is different from p(hp), the integrals end up the same because in one case you integrate with respect to sigma, and the other with respect to c, so when you take into account the Jacobian, the math is identical.

As far as the difference between the STAN output, I am still not sure what is going on, but I think it does relate to the uninformative prior. When I put a gamma prior on hp/hp\_alt, both formulations give the same results.

---

<div class="post-metadata">

**Author:** ![sakrejda](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sakrejda/32/845_2.png) [@sakrejda](https://discourse.mc-stan.org/u/sakrejda)\
**Post date:** [May 25, 2017, 12:10pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/26 "2017-05-25T12:10:00Z")

</div>

> [@aaronjg](#):
>
> As per Michael’s comment, the jacobian is not needed.

Michael was commenting on something different than me and Bob. K

---

<div class="post-metadata">

**Author:** ![sakrejda](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sakrejda/32/845_2.png) [@sakrejda](https://discourse.mc-stan.org/u/sakrejda)\
**Post date:** [May 25, 2017, 6:15pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/27 "2017-05-25T18:15:35Z")

</div>

D’oh, I just had to think this through for a model. Jus of course you’re right! :P

---

<div class="post-metadata">

**Author:** ![JulianK](https://avatars.discourse-cdn.com/v4/letter/j/779978/32.png) [@JulianK](https://discourse.mc-stan.org/u/JulianK)\
**Post date:** [May 31, 2017, 11:35am UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/28 "2017-05-31T11:35:31Z")

</div>

Glad to see that my toy problem is still causing so much consternation (!).

Thanks for looking into it :)

---

<div class="post-metadata">

**Author:** ![ryan-richt](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ryan-richt/32/13349_2.png) [@ryan-richt](https://discourse.mc-stan.org/u/ryan-richt)\
**Post date:** [October 4, 2018, 8:44pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/29 "2018-10-04T20:44:07Z")

</div>

Hey @Bob_Carpenter! I know this is old but since it’s preserved for posterity…  
Does this example (`target += theta3;`) need a Jacobian adjustment _at all_ ? The user manual says:

`Whenever a nonlinear transform is applied to a parameter, such as the logarithm function being applied to beta here, and then used on the left-hand side of a sampling statement or on the left of a vertical bar in a log pdf function, an adjustment must be made...`

Maybe there is one more addition that should go in the manual if the adjustment is necessary for consumption of transformed parameters that never appear in either of these positions?

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [October 8, 2018, 9:43pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/30 "2018-10-08T21:43:18Z")

</div>

For the lognormal, you need the Jacobian adjustment.

It’s a little more subtle than written in the manual given that Stan can be used in lots of different ways. For example, `normal(a | b, c)` is equivalent computationally and mathematically to `normal(b | a, c)`, but they’re conceptually different.

The point of Jacobians is that if you have a distribution p\_X(x) and a smooth, monotonic transform f:\mathbb{R} \rightarrow \mathbb{R} and you want to know the distribution of Y = f(X), then you have:

p\_Y(y) = p\_X(f^{-1}(y)) \cdot \left| \frac{\partial}{\partial y} \, f^{-1}(y) \right|.

In Stan, this means that if you declare

```
parameters {
  real y;
  ...
transformed parameters {
  real x = f_inv(y);
  ...
model {
  target += foo(x | theta); 
  target += log(abs(d_f_inv(y)); // d_f_inv(y) = f_inv'(y)

```

The Jacobian’s necessary here because the distribution’s being defined for `x` with `x ~ foo(theta)`, but we want to get the right distirbution on `y`, which requires the Jacobian adjustment. The `target += foo(x | theta)` gets you the \log p\_X(f^{-1}(y)) term, and the `target += ..` gets the \log \left| \frac{\partial}{\partial y} \, f^{-1}(y) \right| term (everything’s on the log scale in Stan). So now you’ve defined the right log density for `y`, namely

\log p\_Y(y) = \log p\_X(f^{-1}(y)) + \log \, \left| \frac{\partial}{\partial y} \, f^{-1}(y) \right|.

---

<div class="post-metadata">

**Author:** ![ryan-richt](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ryan-richt/32/13349_2.png) [@ryan-richt](https://discourse.mc-stan.org/u/ryan-richt)\
**Post date:** [October 10, 2018, 4:42pm UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/31 "2018-10-10T16:42:51Z")

</div>

Thank you @Bob_Carpenter!

I guess I have scoured the forums and docs and wanted to have a google-able resource for posterity that is this different (?) case. Every complete case I can find in the forums/docs is, as you explained above, a function of a single parameter, used on the LHS. This post from @sakrejda above (that reponded to a year ago) is interesting because he included a Jacobian correction for a parameter used only on the RHS. If your last post applied to any parameter transform, even if used only on the RHS, then why didn’t we also need to add log(abs(theta2)) as well, to get the correct distribution on sigma2\_unit from @sakrejda’s same code snippet?

Thank you again!

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [October 22, 2018, 2:16am UTC](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584/32 "2018-10-22T02:16:18Z")

</div>

It’s a matter of what density you’re trying to compute, not where things show up in the formula—Stan lets lots of things be written different ways.

When you need a Jacobian is when you have a parameter `theta` and a transformed parameter `alpha = f(theta)` and we want to define a density on `alpha` and have it propagate to the proper density on `theta`. I tried to explain that as clearly as I could in the previous post by writing out the long form densities p\_Y(y) for the random variable Y and showing how that can be defined in terms of p\_X(x) given a mapping X \mapsto Y.

[Previous page](https://discourse.mc-stan.org/t/non-centered-parameterization-on-variance-parameter/584.md?page=1)
