A few weeks ago, @wds15 proposed switching ODE integrators between the warmup and the sampling phase. The idea is the following: use the bdf solver during warmup, when the chain moves into more “extreme” regions of the parameter space; but once we’re in the typical set, use a non-stiff solver.
ODE-models are delicate, to say the least, so I struggled a bit to set up a good computer experiment. But here’s a pharmacokinetic example, with code and everything, where the method works quite nicely.
Note that if you make the priors tighter, you can prevent the chains from wandering into pathological regions of the parameter space and the non-stiff solver works just fine. On the other hand, loosen the priors too much, and even the stiff solver fails. This might be worth exploring further.
There may be better examples to learn about switching solvers.
I also played with a Friberg-Karlsson model. Here the system is stiff, and you need to keep using the bdf solver during the sampling phase (at least in the prior / data setup I used).