No you’re not. Math’s infrastructure is not designed for PDE.
I’m glad you’re dropping STALD for the moment. The original STALD paper(I think it’s in the reference of cvode user guide) shows that it’s designed for specific problems (advection-convection) where spurious oscillations could happen when certain BDF methods are used, so it will unlikely benefit problems like the heat equation @jtimonen posted earlier, and most flow solvers nowadays no longer practice this anyway. In addition, by assuming small problems size and using serial vector, Stan’s CVODES and adjoint implementation becomes irrelevant for most real-world PDE problems. Again, it comes down to infrastructure.
IMO the best bet is to interfacing external solvers, as we’ve seen here.
Yup, this is what I figured in that STALD is designed for very specific applications which we do not really target.
Instead I have built in a sparse matrix thing to try out for now. Not sure if it will make it to the exposed thing. Sparse matrix in the sense that the Jacobian is thresholded for very small entries leading to (hopefully) faster LU solves. Right now I can’t find a problem size large enough yet where I see a gain, but I could have messed up coding it… not sure ATM.
Yeah I’ve considered the option for Torsten before quickly (maybe too quick?) turning away. Several factors play here:
The sparsity of the matrix. In general matrices with <50% sparsity will have hard time to gain advantage, though it really depends to the sparsity pattern and
The implementation that matches the pattern. There needs to be a sparse solver and matching storage set up to take advantage of the sparsity, and
The hardware. If the memory bandwidth is sufficient enough for a dense matrix turnout, such as when the matrix is small(<100 x 100 as in most Stan’s ODE problems), dense format will almost always more efficient.
I also want to weigh in on this with my (frankly unqualified) opinion.
Non-toy PDE problems (>1 spatial dimension, >1 field variables, nonlinear problems) would generally need much more infrastructure than Stan currently provides,
such as sparse matrix algebra, iterative solvers with preconditioners or unstructured mesh handling.
Interfacing with external solvers appears much more productive.
Very good question. Such a thing would be of great value as IMO our current test cases are far from sufficient. This post shows the ODEs that are routinely used in development and you can see it’s a short list. There’s a plan to expand it systematically, but first we’ll need to tie some loose ends, such as ODE testing cleanup · Issue #2395 · stan-dev/math · GitHub. Help wanted!
That’s fantastic, thank you! There’s actually an ongoing discussion within @SGB on how we can regularly update our roadmap like what Rust has been doing, to help planning and bringing in new contributors. While this effort is underway, things that come to my mind that might interest you include
A friendly user guidance on how to select ODE solvers and control parameters, e.g. here and here, @bbbales2 will love this.
The advantage to that is it’s something we want to get done for the next release and @wds15 will do most of the legwork.
It is not necessary you catch every problem or anything or spend a million hours reviewing it – just get feedback to @wds15 if something doesn’t make sense or seems weird or doesn’t work (it’s very easy to miss things, and catching things late leads to panic’d releases). Also it’ll all get double checked before it gets merged.
Edit: Well, we want to get it done for next release if it turns out to be a good idea which it seems like at this point it is, but I’ve really not been paying attention which is why people paying attention is useful*
I’m late to the party but I’ve enjoyed reading the discussion so far. Thanks for igniting the conversation @Funko_Unko .
A few weeks ago, @bbbales2 told me about adding recommendations for tuning ODE integrators in the Stan manual. It feels awkward to write recommendations when there are still many unanswered questions – and in certain cases, we still need to work out what the right questions are. Still, there are some useful heuristics and proven strategies, as this thread demonstrates.
Understanding Stan’s underlying mechanics can help avoid pitfalls – does that count as best practices? --, one example can be found in the recent case study on disease transmission. Section 5 resonates with some of the discussion happening here.
There’s also more work on switching ODEs during HMC phases, as teased during the presentation I gave last week and motivated by @wds15 's idea. And other things happening on other fronts.
Might be worth starting a new thread about the prototype adjoint solver. The linked-mass-flow example in the adjoint branch returns divergent transitions, unless I crank adapt_delta to 0.99 (which takes about twice as long to run). Also, it’s worth running things in parallel, so a command like
You should start the example with init=0 then it did work for me at least.
I have already run SBC over this thing and it did come out just fine.
I plan to announce the prototype in a separate post which will hopefully catch even more users attention. We should aim to close the testing phase this month. Then this leaves enough time before mid May we release the next Stan version. This sounds like plenty time to get this thing straight.