As I understood, the kinsol solver is better than the currently available algebra solver and we should make this thing available ASAP, I think. Or were there reasons to not expose this to the Stan language?
Given that it’s not yet exposed, we should probably first change it to a variadic interface to let users work with it much more conveniently.
In all the examples I looked at, the Kinsol solver is either better or much better than Powell. But there may be counterexamples worth documenting. Nothing prevents us from exposing Kinsol to the language; but based on the feedback I received, it didn’t seem too high on the priority list.
We can hold off until the variadic interface is available and I can then write the relevant PR.
To answer your question: I’m not aware of any work being done on the variadic interface. @bbbales2 any further insight?
Issue sounds great. It’s not on you - @bbbales2 - to make stan math variadic…
@charlesm93 would it make sense to go with different names compared to what we have? How about we go with
algebra_solver = current powell
algebra_solver_powell = current Powell
algebra_solver_newton = kinsol
is that sensible? This would resemble the integrate_ode logic.
EDIT: I remember that you, @charlesm93, also wanted to improve the performance of these solvers given that we now can do AD during the reverse sweep. Is that recorded in an issue and are you still on that?
Sorry, for pushing this, but I do have bad experience with Powell and would love to see something much better land in Stan.
@wds15 I agree with the name convention you propose.
Sorry, for pushing this, but I do have bad experience with Powell and would love to see something much better land in Stan.
Sorry to hear that. I was under the impression there was very little demand for the Kinsol solver. I don’t have experience with the variadic arguments. If we want the Kinsol solver in Stan’s upcoming release, I can write the relevant stanc3 code for the current function: it should be a carbon copy of the code for the Powell solver and maybe the two functions can share the same code (as was done for the ode integrators in stan).
If we want the variadic arguments, I have to check how much work this requires.
There are multiple ways to improving the differentiation. There is an old issue in stan-math, but a more serious approach is described in this discourse post. The key feature is the nested autodiff during the reverse-mode sweep that @bbbales2 implemented.
This sounds tempting! Though we should probably change to the naming convention in stan-math first to what I suggested?
The variadic interface will be some work to do as it requires fancy C++ coding, but there is already a variadic reduce_sum and a variadic ode solver interface available… it’s not too hard, but quite some work to plumb the C++.
Changing the kinsol_solver to algebraric_solver_newton and then expose that thing as is will be much faster… would be fine for me.
(for the variadic interface we would need new names for the functions as these must be different)
What’s the policy for exposing new functions? Are we supposed to expose them to both stanc2 & stanc3 or just stanc3 would suffice? @mitzimorris@rok_cesnovar
new functions are not being back-ported to stanc2.
eventually we will stop including stanc2 in the release; right now, it’s there mainly to help debug the stanc3 compiler; if a user has a program that did compile and run under stanc2, either the stanc2 parser was too lax, or it’s a bug in stanc3. we’ll fix the latter.
OK. I can arrange today before the Stan meeting (10am your time) or tomorrow on Friday the same time. Just let me know what’s better. The only constraint I have is that I have to leave today at 10:40am for another appointment, but 30min should do, I hope.