Centered vs noncentered -

General
1

Heyo!

I am still learning and unsure about centered vs noncentered modelling. I have a simple linear regression model and am trying to understand the modelling with this before I move on to complex stuff.
I had

model {
//// priors
beta_0 ~ cauchy(0, 5);
beta ~ cauchy(0, 5);
sigma ~ gamma(0.001, 0.001);
//// likelihood
price ~ normal(beta_0 + x * beta, sigma);
}

for now and was wondering if the correct way to non-center this would be:

transformed parameters{
vector[n_obs] price_mean;
price_mean = beta_0 + x * beta;
}

model{
//// priors
beta_0 ~ cauchy(0, 5);
beta ~ cauchy(0, 5);
sigma ~ gamma(0.001, 0.001);
//// likelihood
price ~ normal(price_mean, sigma);
}

I feel that I am not completely grasping something here. Would this already be “reparametrizing”? I feel that I only gave something a different name - then again, reparametrizing can be just that.

edit: Just for completeness:
price and beta are vectors, beta_0 is a real and x is a matrix. All have the right dimensions, both models run and produce the same results, I would say. As this is not a complex thing, I had expected that and was only looking to get the gist of it.

edit2:
After thinking about it, I would say this is not about non-centering, as I did not split anything up in smaller parts - is that correct?

2

Yeah what you’ve done here is not a reparameterization. What you did is to store the result of beta_0 + x * beta in a variable called price_mean. So you’ve written the same model using slightly different Stan code but it’s not a reparameterization.

The non-centered parameterization that you refer to can be used when you have particular kinds of hierarchical priors. So in your case, imagine you had

model {
...
beta ~ normal(mu_beta, sigma_beta);
}

where mu_beta and sigma_beta are parameters in the model. In that case you would be using a “centered” parameterization for beta because the prior is centered around the prior mean mu_beta. However, you could write a statistically equivalent model using the “non-centered” parameterization

parameters {
  ... 
  real beta_raw;
}
transformed parameters {
  ...
  real beta = mu_beta + beta_raw * sigma_beta;
}
model {
  ...
  beta_raw ~ normal(0, 1); // implies beta ~ normal(mu_beta, sigma_beta)
}

in which case the prior for beta_raw is standard normal and then we compute beta by scaling beta_raw by sigma_beta and shifting it by mu_beta. That is, we take a normal(0, 1) and scale it by sigma_beta to get normal(0, sigma_beta) and then shift it by mu_beta to get normal(mu_beta, sigma_beta).

I’m not saying this necessarily makes sense to do in your case, just trying to use your example.

I highly recommend @betanalpha’s case study at Diagnosing Biased Inference with Divergences, which shows an example of centered and non-centered parameterizations and discusses when this type of reparameterization can be useful.

2 Likes
3

Thank you! Yeah, I was unsure regarding this, but I agree that this is no reparametrization. And thank you for the example! I feel I am understanding it better and better the more I read and work with it - which is obviously not a surprise. ;)

4

This is all written up in the manual, along with a lot of other examples.

5

You’re right! I had read that, but was still unsure. (Now, a couple of days later, I believe I understand it a lot better)

I am getting more and more into Stan and also more and more in the ability to judge the scope of things I do not understand. Thanks for all the support here!

1 Like