When I was writing the non-identifiability blog post, I really tried to think hard where divergences are expected to occur. And while the divergences somewhat gathered in the areas I expected them too, they were also often marked quite far from those area. I noticed that the Stan manual says:
ShinyStan provides visualizations that highlight the starting point of divergent transitions to diagnose where the divergences arise in parameter space.
So this means that if an iteration is divergent, Stan keeps the location in parameter space where the iteration started, possibly multiple integration steps before the actual divergence occurred? If this is correct interpretation, would it be possible to export the locations of the start of the integration step that diverged? In addition, the gradient at this point (or, equivalently, the attempted step) could be stored. My best guess is that this would be a more illustrative diagnostic when overlaid over the plots of the samples, clearly outlining regions with problematic geometry.
Typical example is this pairs plot:
I strongly suspect that divergences actually occur at the boundary of the sampled region as I expect the curvature there to be much higher than in the middle of the sampled region (this is from the sigmoid model that does have only the two parameters shown).
If you don’t see any problems with this idea, I might try to add it to Stan. I imagine having a separate, optional stream for just this data.