Yeah I’ve considered the option for Torsten before quickly (maybe too quick?) turning away. Several factors play here:
The sparsity of the matrix. In general matrices with <50% sparsity will have hard time to gain advantage, though it really depends to the sparsity pattern and
The implementation that matches the pattern. There needs to be a sparse solver and matching storage set up to take advantage of the sparsity, and
The hardware. If the memory bandwidth is sufficient enough for a dense matrix turnout, such as when the matrix is small(<100 x 100 as in most Stan’s ODE problems), dense format will almost always more efficient.
I also want to weigh in on this with my (frankly unqualified) opinion.
Non-toy PDE problems (>1 spatial dimension, >1 field variables, nonlinear problems) would generally need much more infrastructure than Stan currently provides,
such as sparse matrix algebra, iterative solvers with preconditioners or unstructured mesh handling.
Interfacing with external solvers appears much more productive.
Very good question. Such a thing would be of great value as IMO our current test cases are far from sufficient. This post shows the ODEs that are routinely used in development and you can see it’s a short list. There’s a plan to expand it systematically, but first we’ll need to tie some loose ends, such as ODE testing cleanup · Issue #2395 · stan-dev/math · GitHub. Help wanted!
That’s fantastic, thank you! There’s actually an ongoing discussion within @SGB on how we can regularly update our roadmap like what Rust has been doing, to help planning and bringing in new contributors. While this effort is underway, things that come to my mind that might interest you include
A friendly user guidance on how to select ODE solvers and control parameters, e.g. here and here, @bbbales2 will love this.
The advantage to that is it’s something we want to get done for the next release and @wds15 will do most of the legwork.
It is not necessary you catch every problem or anything or spend a million hours reviewing it – just get feedback to @wds15 if something doesn’t make sense or seems weird or doesn’t work (it’s very easy to miss things, and catching things late leads to panic’d releases). Also it’ll all get double checked before it gets merged.
Edit: Well, we want to get it done for next release if it turns out to be a good idea which it seems like at this point it is, but I’ve really not been paying attention which is why people paying attention is useful*
I’m late to the party but I’ve enjoyed reading the discussion so far. Thanks for igniting the conversation @Funko_Unko .
A few weeks ago, @bbbales2 told me about adding recommendations for tuning ODE integrators in the Stan manual. It feels awkward to write recommendations when there are still many unanswered questions – and in certain cases, we still need to work out what the right questions are. Still, there are some useful heuristics and proven strategies, as this thread demonstrates.
Understanding Stan’s underlying mechanics can help avoid pitfalls – does that count as best practices? --, one example can be found in the recent case study on disease transmission. Section 5 resonates with some of the discussion happening here.
There’s also more work on switching ODEs during HMC phases, as teased during the presentation I gave last week and motivated by @wds15 's idea. And other things happening on other fronts.
Might be worth starting a new thread about the prototype adjoint solver. The linked-mass-flow example in the adjoint branch returns divergent transitions, unless I crank adapt_delta to 0.99 (which takes about twice as long to run). Also, it’s worth running things in parallel, so a command like
You should start the example with init=0 then it did work for me at least.
I have already run SBC over this thing and it did come out just fine.
I plan to announce the prototype in a separate post which will hopefully catch even more users attention. We should aim to close the testing phase this month. Then this leaves enough time before mid May we release the next Stan version. This sounds like plenty time to get this thing straight.
Furthermore, for the way I set up my toy models, the following worked out exceedingly well, significantly reducing walltime and runtime differences between chains and increasing ESS:
Instead of a single run using all of the data, warm the model up in chunks. I.e. take 1,2,4,8,16,… timesteps, and for each run perform a short warmup phase with adapt_init_phase=25, adapt_metric_window=100, adapt_step_size=0, iter_warmup=125 and initialized with a metric computed from the (relevant) warmup samples from all chains from the previous run. For the final run first do the above, compute the metric, and then do the sampling preceeded by the standard window to adapt the step size.
Is there anything subtly or not so subtly wrong with the above?
The need for a dense metric is to me a hindsight for bad parametrization as it suggests that the posterior is quite correlated which can be avoided once reparametrized (not always possible, nor do I have the time to do that for every problem).
Can you explain a bit more your warmup strategy? What do you mean by timesteps here? Can things run in parallel from what you describe?
Learning with short runs on warmup and then expand on that can make sense; but this is actually also done by the warmup procedure itself (which is not necessarily the best in the world, but certainly a very robust procedure…though we all wish for it to run faster which is hard)