Why multiplying the lpdf by a constant is invariant while dividing leads to failure?

I am aware that doing operations on lpdf is not the standard way to proceed (and that must be a trivial reason to this), but I am curious why multiplying the lpdf by a constant is invariant

``````x = rnorm(100, 4, 3);

stan(model_code="
data{ int N;
real x[N];}
parameters{
real mu;
real<lower=0> sigma;
}
model{
target += normal_lpdf(x | mu, sigma) * 10000;
}
",
data =list(N = N, x = x))
``````
``````             mean se_mean   sd        2.5%         25%         50%         75%       97.5% n_eff Rhat
mu           3.69    0.00 0.00        3.69        3.69        3.69        3.70        3.70   981    1
sigma        3.07    0.00 0.00        3.07        3.07        3.07        3.07        3.08  4557    1
lp__  -2541121.85    0.03 0.97 -2541124.40 -2541122.24 -2541121.55 -2541121.14 -2541120.88  1496    1

Samples were drawn using NUTS(diag_e) at Thu Sep 05 22:06:04 2019.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at
convergence, Rhat=1).
``````

While dividing by a constant leads to failure

``````stan(model_code="
data{ int N;
real x[N];}
parameters{
real mu;
real<lower=0> sigma;
}
model{
target += normal_lpdf(x | mu, sigma) / 10000;
}
",
data =list(N = N, x = x))
``````
``````                mean      se_mean           sd          2.5%           25%            50%            75%          97.5%
mu     -5.309213e+12 5.435324e+12 9.125369e+12 -2.718509e+13 -6.702003e+12  -3.620251e+10   5.901567e+10   1.286364e+11
sigma  1.137980e+305          NaN          Inf  1.823595e+20  7.674246e+84  9.643890e+158  2.100146e+232  2.716923e+299
lp__   -9.200000e-01 0.000000e+00 0.000000e+00 -9.200000e-01 -9.200000e-01  -9.200000e-01  -9.200000e-01  -9.200000e-01
n_eff Rhat
mu        3 2.47
sigma   NaN  NaN
lp__   3503 1.00

Samples were drawn using NUTS(diag_e) at Thu Sep 05 21:55:43 2019.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at
convergence, Rhat=1).
``````

Thatâ€™s not whatâ€™s happening. Itâ€™s not actually invariant, even though it may seem that way.

If you take a look at the standard error for the mean of the parameters, `se_mean`, youâ€™ll see that itâ€™s going to 0.

What your model is doing, if you multiply by 10000, is youâ€™re saying youâ€™ve seen your data 10000 times. So, your estimates for `mu` and `sigma` should be really good. Itâ€™s equivalent to this:

``````for (j in 1:10000)
target += normal_lpdf(x | mu, sigma);
``````

And just to be absolutely clear, the model is saying youâ€™ve seen the whole `x` array 10000 times, so this is equivalent:

``````for (j in 1:10000)
for (n in 1:N)
target += normal_lpdf(x[n] | mu, sigma);
``````

When you divide by 10000, itâ€™s doing the opposite of thisâ€¦ which is sort of saying that youâ€™ve seen 1/10000 of the data you actually have. (Thatâ€™s not quite right, but hopefully itâ€™s illustrative enough.)

7 Likes