Simple Example of an Improper Posterior Without Warnings from Stan

[maxbiostat]

@betanalpha

I’m not sure what additional/stronger warnings you would expect.

The point is not what I would expect, but rather that sometimes diagnostics fail. This is a moderately interesting instance of that. You don’t have to agree; you may very well think it’s silly.

@betanalpha

The treedepth indicates very long trajectories which could be due to small stepsizes or an ill-posed target density, and following up by examining the unconstrained samples here one can confirm that it is the former.

Good point. But I don’t think you can expect practitioners to look at samples in the unconstrained space. Hence, the absence of divergences or other stronger warnings could lead someone less experienced to think things were fine, or at least that increasing tree depth would solve the problem [doing that will randomly “solve” the problem].

@betanalpha

For additional evidence the quantiles for a are all pushed up against 1 which indicates problems.

Disagree. If I do:

parameters {
  real<lower=0,upper=1> a;
}
model {
  a ~ beta(9999, 1);
}

I also get all quantiles pushing against 1.

@betanalpha

The bulk ESS warning, however, suggests that the mean is ill-defined, which further supports with an ill-posed target.

Except you don’t always get that warning, as the OP found and I confirmed in different versions.

So, to sum up as this is the last I’ll say on this matter: both OP and myself thought this was a weird little example of a target that’s clearly ill-posed and yet might escape detection by diagnostics. And I think this is something that could be emphasized more. I may very well be misrepresenting here, but I think this was kind of @anon75146577’s point here as well – “diagnostics can and in fact often do fail”. Please do let me know if I misunderstood.