Why multiplying the lpdf by a constant is invariant while dividing leads to failure?

I am aware that doing operations on lpdf is not the standard way to proceed (and that must be a trivial reason to this), but I am curious why multiplying the lpdf by a constant is invariant

x = rnorm(100, 4, 3);

stan(model_code="
   data{ int N;
      real x[N];}
   parameters{
      real mu;
      real<lower=0> sigma;
   }
   model{
      target += normal_lpdf(x | mu, sigma) * 10000;
   }
     ", 
data =list(N = N, x = x))
             mean se_mean   sd        2.5%         25%         50%         75%       97.5% n_eff Rhat
mu           3.69    0.00 0.00        3.69        3.69        3.69        3.70        3.70   981    1
sigma        3.07    0.00 0.00        3.07        3.07        3.07        3.07        3.08  4557    1
lp__  -2541121.85    0.03 0.97 -2541124.40 -2541122.24 -2541121.55 -2541121.14 -2541120.88  1496    1

Samples were drawn using NUTS(diag_e) at Thu Sep 05 22:06:04 2019.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at 
convergence, Rhat=1).

While dividing by a constant leads to failure

stan(model_code="
   data{ int N;
      real x[N];}
   parameters{
      real mu;
      real<lower=0> sigma;
   }
   model{
      target += normal_lpdf(x | mu, sigma) / 10000;
   }
     ", 
data =list(N = N, x = x))
                mean      se_mean           sd          2.5%           25%            50%            75%          97.5%
mu     -5.309213e+12 5.435324e+12 9.125369e+12 -2.718509e+13 -6.702003e+12  -3.620251e+10   5.901567e+10   1.286364e+11
sigma  1.137980e+305          NaN          Inf  1.823595e+20  7.674246e+84  9.643890e+158  2.100146e+232  2.716923e+299
lp__   -9.200000e-01 0.000000e+00 0.000000e+00 -9.200000e-01 -9.200000e-01  -9.200000e-01  -9.200000e-01  -9.200000e-01
      n_eff Rhat
mu        3 2.47
sigma   NaN  NaN
lp__   3503 1.00

Samples were drawn using NUTS(diag_e) at Thu Sep 05 21:55:43 2019.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at 
convergence, Rhat=1).

That’s not what’s happening. It’s not actually invariant, even though it may seem that way.

If you take a look at the standard error for the mean of the parameters, se_mean, you’ll see that it’s going to 0.

What your model is doing, if you multiply by 10000, is you’re saying you’ve seen your data 10000 times. So, your estimates for mu and sigma should be really good. It’s equivalent to this:

for (j in 1:10000)
    target += normal_lpdf(x | mu, sigma);

And just to be absolutely clear, the model is saying you’ve seen the whole x array 10000 times, so this is equivalent:

for (j in 1:10000)
  for (n in 1:N)
    target += normal_lpdf(x[n] | mu, sigma);

When you divide by 10000, it’s doing the opposite of this… which is sort of saying that you’ve seen 1/10000 of the data you actually have. (That’s not quite right, but hopefully it’s illustrative enough.)

7 Likes