Constrained prior and posterior projection

Modeling
1

Hi, everyone, I found something interesting. In a single index model, for example,

y=g(\beta^T X) + \epsilon,

where g is unspecified, we might meet constrained parameters that ||\beta||=1 for model identifiability. In Stan, it is usually achieved by

parameters {
  unit_vector[p] beta;
} 
model{
beta ~ N(0, 1);
}

to define a uniform prior on the hypersphere. However, I found it may lead to a large bias to the estimation of \beta. An alternative is discussed by [1812.05741] Constrained inference through posterior projections where they discuss the posterior projection and we may code like

parameters {
  vector[p] beta;
} 
model{
beta ~ N(0, 10000);
}
generated quantities{
unit_vector[p] beta_pro;
beta_pro = beta ./  sqrt(sum(beta .* beta));
}

and I found this way may significantly reduce the bias.

That’s quite interesting. Can anyone give some explanation to it? Many thanks.

2

Can you clarify what you mean by bias? Can you also clarify why you choose to put a standard normal prior on beta (assuming this is what you mean by N(0,1))? This will interact with the unit_vector constraint in weird ways.

Is it perhaps the case that there’s some confusion arising from the Stan Users Guide page on Unit Vectors, where it mentions that each of the unconstrained variables that underlies Stan’s unit_vector implementation is given a standard normal adjustment. This is handled automatically and under-the-hood on the unconstrained scale in Stan, and does not need to be done by the user in the model block.

3

Thank you. The bias means that if I have a true value of \beta_0, the bias between it and the posterior mean.

As the User Guide mentions, Stan realize the unit vector by reparameterization and the prior N(0, 1) leads to a uniform prior on the hypersphere. I also tried other noninformative prior such as N(0, 10000) but it does not change the result much.

4

This must occur in the context of some particular model and data. Can you provide a reproducible example of this bias occurring?

This is handled automatically and under-the-hood in Stan. If you want your prior to be uniform over the hypersphere, then don’t add any prior statements. When you declare a unit_vector, Stan automatically gives it a uniform prior over the hypersphere.

5

Thanks. Well, I am still working on the related problem and I will update once I submitted for review.