Was curious to understand the issue a bit more, the function I wrote:
stan_checkdivergences.R (4 KB)
seems to do a decent job on the gp and 8 schools examples, when using the relative location of divergences as a guide… not sure whether I’m just getting lucky / overfitting though, as I expected the sd (function returns both) to be more useful :)
for gp:
$locationmeans
sigma “-0.445589299588228”
z[4] “-0.35245496811889”
z[9] “-0.245067074753998”
z[1] “-0.187424643926895”
z[5] “-0.0615538808042705”
z[10] “-0.0606798964844923”
z[6] “0.0259601372553438”
z[7] “0.0183496503172226”
z[2] “0.0164146804177184”
z[3] “0.0160371623666628”
z[8] “0.0108553379725488”
l “0.0104059696918443”
and 8 schools
$locationmeans
tau “0.500108302039656”
theta[5] “-0.112244942837985”
mu “-0.0993788081711008”
theta[3] “-0.0965787075914174”
theta[6] “-0.0694263357149289”
theta[7] “0.0549598237222587”
theta[8] “-0.0437538694385667”
theta[4] “-0.0403247562380711”
theta[2] “0.00166683440376185”
theta[1] “-0.000197107611385944”
Hi, there’s so much in this thread that I’m not sure what I’m supposed to be blogging. So if anyone wants to put this in the blog, that’s fine with me.
A
I pasted the plots into the blog as part of the Stan weekly report and linked back to this thread, so it’s already done.
Thanks for this Charles, I just tried it for finding divergences in one of my models. I haven’t tried the parellel coordinates plot yet, maybe it it will point out the same thing. This identifies a funnel between two of my params as the lowest sd.
I added a feature to the function to specify (either by name or by numeric index) which params you want to look at. This was modified from the original version posted above but would be trivial to integrate into the new version. I also added a composite score ranking (rank the sd and loc and multiply them together.)
stan_check_divergences_ch.R (2.8 KB)
1 Like
Thanks for reporting back! I think I also tried the composite rank, but must have been doing it manually afterwards…
No problem. I’m actually thinking of writing up a blog post on debugging my model, would you mind if I reference my modified version as a gist on github? I’ll attribute in the blog post, and if you like I can put you in the author tag in the roxygen block.
Sure, sounds good. If you’re looking into large, slow, troublesome models, the stanWplot (stan with online trace plots) function I wrote (can find it as part of the ctsem package on github) might also be helpful.