I’m really not the expert, but to me it sounds like your chains may occasionally get stuck in (unphysical) metastable wells in the posterior landscape, which to add insult to injury stress the numerical integrators due to excessive stiffness, resulting in the large runtime variations.
See this thread Fitting ODE models: best/efficient practices? and most importantly the first link in the first post linking to a case study by @charlesm93. Maybe the steps taken in the case study can help you?
Edit: Ah wait, but you say you initialize your chains near the correct values?