Sampler Diagnostics

Great work! I am intrigued.

The model is described here: Identifying non-identifiability

Not sure if you found that in the previous thread, but this is not correct - the new point is sampled from the points built up in the last doubling that did not include divergences. Also this way of choosing the points maintains some desirable properties of the sampler so that it may (although rarely actually does) sample the correct posterior even when facing divergences. Due to this point chosen by your heuristic cannot be returned as the single point from the divergent iteration.

Michael always reminded us that the whole trajectory is important. I guess your heuristic to choose one point is better than the one I proposed, but it might make sense to make the whole trajectory available. Nevertheless, I am personally heavily for having something good soon than something perfect in an uncertain future.

Right now I see great value in you just playing around with this, but should you eventually have ambition to push this into Stan core I should warn you there are some potential roadblocks that may take quite some time and effort to overcame, so please incorporate this into your expectations.

When I attempted this, the biggest block against including additional divergence diagnostics in Stan when I was playing with it was that it required a refactor of the output functionality, which we discussed at Proposal for consolidated output but then the discussion didn’t really get anywhere, we all burned out on it and AFAIK the refactor never happened.

But that’s far future: the project is definitely worthwhile on it’s own (might even end up as a nice little paper by you or something) and if it proves to work well, we’ll figure this out!

Best of luck!

1 Like