As we talked about in the meeting, the statistical tests don’t really examine the fact that the spikes are often at 0, but they’d probably pass in these cases.
I made two more pdfs of charts that I think might show that the
fabs issue was likely a red herring - there’s a linear regression with while loop (lin_regr, model) and with fabs (lin_regr_fabs, model)
lin_regr_fabs.pdf (23.7 KB)
lin_regr.pdf (24.0 KB)
<= always on the left,
> always on the left in these)
You can see they aren’t the same, especially on the most granular histograms, but they might be close enough? I don’t have a good intuition for this.
It also seems like the issue exists roughly equally in both
The spikes aren’t that weird by themselves, but the fact that one shows up at 0 in like 7 out of 8 histograms of appropriate granularity across seeds and various ways of getting sigma seems weird to me…