Blog post: Identifying non-identifiability

This is really cool. In the spirit of giving back. The problem you identify in the sigmoid model with non-identified special case made me think of the problem with multi-collinearity. That is, a parameter blowing up because of near non-identifiability. In one of Andrew Gelman’s papers there was some suggestion to solve that with a sort of joint prior for some of the parameters. I had some success with taming divergences in that way but I have no idea whether there are any problems with the approach. I think for your model it would like this.

w ~ normal(0, prior_sd);
b ~ normal(0, prior_sd);
prior_sd ~ normal(0, 2.5); // Or whatever is appropriate in your case.

Again, intuitively it makes sense for me to have a prior that says if w is not very large, it is unlikely that b is very large (in this case) but that is also the only motivation I have.

1 Like