Compromize for default parameterization in a package

Developers
1

Hi all,

I’m developing a package for TKTD models (Toxicokinetics/Toxicodynamics). My question is about default parameters (number of iteration, adapt_delta, ODE…), the compromise between robustness and speed.

I have around 15 data set to test my package.

  • For instance, with 80% of them, adapt_delta=0.8 work fine, but for other I need adapt_delta=0.95 and for one of them adapt_delta=0.99. If I’m right, ‘rstan’ default is 0.8 and ‘rstanarm’ default is 0.95. I didn’t see big different in speed for my datasets, so I’ll probably go for 0.95, and document the case where 0.99 is required.
  • For ODEs parameters, the stuff is more complex. I use tolerance parameters at ‘10e-8’, ‘10e-5’, and ‘1e3’, which work fantastic for most of the data set, but I needed to increase at ‘10e-10’, ‘10e-8’, and ‘1e3’ for 3 of them. And this increases the time to compute. So I don’t want it as default, but understanding the way to change those parameters is more complex for a user I guess.
  • Also, _rk45 work great for all my examples, but I’m not sure if a specific dataset could work better with _bds. Here, the speed matters a lot.

I understand that, by definition, there is no good answer. But what are the kind of questions I should ask myself when developing the package? And what kind of message I should present to the user?

Thank you.
Virgile

2

I would say to go in the direction of rstanarm in the sense that your package defaults should be set to work decently for (almost) all datasets that could be passed to your model fitting functions but pass on options to override the defaults when the user specifies those. The rstanarm package does not currently have ODEs, but I would default your tolerances to 10e-10, 10e-8, and 1e3 and document how to change them. If your Stan program, you can have stuff like

if (stiff) {
  // don't use rk45
} else {
  // use rk45
}

where stiff is defined in the data block and passed from R. Here I think not-stiff is an okay default, but document how to switch it.

3

Ok, I see. Thank you. I’m going to provide the maximum options and set default parameters for maximizing the robustness of fitting. Well, it makes sense.