What does it mean if inference is only sometimes working?

I saw the fit failing to converge 1 out of 5 times with N = 20 and most of the time with N = 200, so I do not think the failures you saw are a fluke.