I don‘t think that the state of affairs is so bad, but here is what I learned so far:
- Avoid ODEs if you can! For example, in PK/PD systems we input drug using functions which themselves are analytically solvable or you can make - for convenience - the input function being represented as a ODE. Don‘t enlarge the ODE system unless you have to!
- Avoid to have more parameters than necessary. Sounds simple, but it is important to keep the number of ODEs being solved down (for each parameter you end of solving N more ODEs if N are the number of states).
- Should your solution approach 0 then definitely tune the absolute tolerance to values large enough so that the ODE solver does not derail and for your problem setting to still make sense.
- The relative tolerance should be increases as well in a similar way.
- Don‘t make the tolerances larger than 10^-4 is what I found in my problems.
- A good resource for tuning tips of the tolerances is the CVODES user manual
- Scale the states such that unity is a sensible number (step-size tuning starts off with this number as a reference scale basically)
- I found in PK/PD problems that using an adapt_delta of 0.7 (so rather low) gave me improved performance. The net effect of that are large step-sizes which did help move the sampler around more quickly.
- Know your problem well in the sense that non-linear systems can easily be quite nasty and cause bi-modality. You need to avoid that by using clever parametrisations and good priors. This is the usual advice to have a favorable „geometry“ of the problem, but it is even more important in ODE problems.
- I did myself in the past consider low max_num_steps a good thing to stop the sampler from going into crazy parameter regimes, but I changed my mind by now. It‘s far better to let the sampler get the value of the log-lik at these extreme values which will let the sampler „know“ that this is no-mans land.
- Using a stiff sampler for warmup (bdf) and then switch for the sampling phase to a non-stiff solver (RK45) has been shown to be good for some problems. In fact, you can actually switch to the non-stiff solver starting with Phase III of the warmup.
That‘s just a collection of my personally collected experience with these problems.
If you are aiming to solve large problems with many states and/or parameters, then you should be looking for the adjoint ODE solving method for which I am about to share an experimental version of it, but it will still take some moments.
I hope that helps.