Built-in constrain vs user transformation

Modeling
1

Hi all,

I have a very basic question for constrained parameter/transform parameters (e.g. lower>0)

Should there be any different in terms of the parameter posterior between sampling from unconstrained space and then do transformation (exp in the case of >0) and directly sampling with constrained parameters?

data {

}

parameters {
  real<lower=0> x_con;
  real          x_unc;
}

transformed parameters {
  real<lower=0> x_trn;

  x_trn = exp(x_unc);
}

model {
  x_con ~ normal (0,1);
  x_unc ~ normal (0,1);
}

This is what I get:

       mean se_mean   sd  2.5%   25%   50%   75% 97.5% n_eff Rhat
x_con  0.80    0.01 0.59  0.05  0.33  0.68  1.13  2.21  2455    1
x_unc  0.04    0.02 0.99 -1.88 -0.62  0.04  0.70  1.97  1802    1
x_trn  1.71    0.05 2.27  0.15  0.54  1.04  2.01  7.18  1855    1
lp__  -1.59    0.03 1.07 -4.39 -2.04 -1.28 -0.82 -0.53  1128    1

It seems that x_con (directly “sampling” of constrained variable) and x_trn (“sampling” from unconstrained and then transform) yield 2 different posterior.

In addition, Stan complained about a few (<10) divergent transitions. Why would this happen?

Many thanks,
Andy

2

As it should. x_con has a truncated normal posterior distribution (i.e. Normal without the negative values)
whilst x_trn has a log-normal distribution due to the exponential transformation. See :Log-normal distribution - Wikipedia

See https://onlinecourses.science.psu.edu/stat414/node/157/ or any statistical inference text (I used Casella and Berger) for details about changes of random variables.

Tough to say, how are you calling Stan? There probably shouldn’t be with sufficient warmup iterations. Might also be due to the lack of data.

3

Also check out Chapter 35, “Transformations of Constrained Variables” and page 80, “User-Transformed Variables” in the 2.17.0 Stan manual. That has more info on exactly the question you’re asking.

When you specify the constraint in the parameters block, extra terms get secretly added to the target log density. When you specify constraints in the transformed parameters block, no extra terms are added or variables transformed. Those constraints are just checked for correctness at runtime (to help you avoid errors).

4

Thanks everyone!

Re divergent transition - I am using rstan: xx <- sampling (model)

5

Yes, they’re totally different models.

x_con will have a half-normal distirbution, p(x^{\mathrm{con}}) = \mathsf{Normal}(x^{\mathrm{con}} \mid 0, 1) / 2

x_unc will have a standard normal distribution, p(x^{\mathrm{unc}}) = \mathsf{Normal}(x^{\mathrm{unc}} \mid 0, 1)

x_trn will have a lognormal(0, 1) distribution, p(x^{\mathrm{trn}}) = \mathsf{LogNormal}(x^{\mathrm{trn}} \mid 0, 1).