Yeah, I agree with this. I’ve certainly been disappointed after making plots of divergences. Maybe there is something more to be had.
I liked the plots @martinmodrak made here: Getting the location + gradients of divergences (not of iteration starting points) . I think he was getting at the same idea.
@betanalpha makes the point in that thread that divergence-ness is a property of the whole trajectory, not just a point on that trajectory.
I think there were IO issues with getting stuff out of actual Stan. Martin managed it somehow.
My buddy and I have an R version of the NUTS algorithm Stan uses that might make it easier to test this (the algorithm is in R but uses Stan gradients). I dunno. It’ll be slow, for sure, but maybe it’s useful at all: https://github.com/bbbales2/RNUTS/blob/master/rnuts_example.R . A little more info on using it here: NUTS misses U-turns, runs in circles until max_treedepth - #31 by bbbales2. If you’re interested I can tell you more.