As far as we can tell, Stan 2.09 is the latest version of Stan with a properly functioning sampler. Versions from 2.10 on are producing biased samples that slightly underestimate posterior variance. Thanks to Matthew R. Becker for filing the issue: https://github.com/stan-dev/stan/issues/2178 …

Let me temper the panic by saying that the bias is relatively small and affects only variances but not means, which is why is snuck through all our testing and application analyses. Ultimately posterior intervals are smaller than they should be, but not so much that the inferences are misleading an…

I updated the blog post with Michael’s comment, which pretty much matches what Andrew said. I’m still mortified, not because bugs get through, but because this is one we should be able to catch. On the plus side, we now have the model to catch this in future regression tests (what computer scie…

From rstan you can set algorithm=“HMC” when calling stan(), but I would trust Michael on this. That is, even with the bug NUTS should be better than static HMC (except for some trivial cases).

@betanalpha , I saw the unit tests. I don’t remember if there was a test added to help us catch this sort of bug in the future. If we didn’t add one with the pull request, could we add one now?

Look at the diff – two tests were added to catch the particular bugs addressed in the PR.

I remember those tests. I was thinking something end-to-end. We know what’s correct in analytic models. We should know if we introduce something that’s not correct.

Stan 2.10 through 2.13 have broken samplers

Developers

betanalpha December 22, 2016, 5:50pm 5

Correct.

Topic		Replies	Views
Old NUTS? Developers	11	1208	December 29, 2016
Choice of sampler General	3	551	December 11, 2018
How to evaluate samplers for inclusion in Stan? Developers	15	866	March 8, 2025
HMC (jittered) vs. NUTS on 1000-dimensional standard normal Algorithms mcmc	9	4271	April 29, 2019
How to produce unbiased samples with HMC method Modeling	8	1018	July 22, 2018

Stan 2.10 through 2.13 have broken samplers

Related topics