I don’t read the forums frequently enough to make a 2d deadline!
I have some questions about scope:
- Will it implement log density and gradients efficiently?
- Will it handle multiple different users/models concurrently?
- Will it manage any kind of persistence?
- Are there performance targets w.r.t. CmdStan or PyStan or RStan?
If the answer to (1) is positive, it rules out CmdStan-based solutions. If the answer to (1) is negative, I don’t see the point of having a server vs. just calling CmdStan (or better yet, a suitable replacement) directly.
If the answer to (2) or (3) is positive, that’s a lot of state to manage. (3) is related to (2). From what I can understand of the current httpstan interface, it persists model IDs.
Python / HTTPStan
It looks like the calls to services just wait to return until the call has completed.
Is the protobuf format documented somewhere? The web service doc only says it returns a binary and doesn’t link to the format.
It looks like this persists model IDs and also IDs for output. How long do those persist?
How is scheduling handled for concurrent requests?
Every language has the infrastructure to provide HTTP or direct socket protocols. How does Julia manage mulitple nodes on a cluster?
Stan itself has the ability to distribute computation of the log density and gradient over clusters of nodes for within-chain parallelism using MPI (multiple nodes or cores) or mulit-threading (multiple cores).
@dlakelan: Are you offering to help build this or just providing suggestions? I’m asking not to discourage suggestions, but because a large part of this decision has to be based on having both the expertise and dedication on the dev team that builds it.
I’ve been working in dynamic languages so long that I don’t even know what people use instead of
make these days
Large projects require more automation and dependency management than a single dynamic language like Julia can provide. This is especially true when you have to link in external resources like Fortran or C++ libraries, documentation generators, testing frameworks, web services, etc. It may be that Julia has its own set of make-like functionality for this, but something needs to figure out even for the purely dynamic part of the language how to rebuild doc when components change. Java built a lot of this into their compiler and their javadoc tooling. But even there, every project I was ever associated with used Ant to organize its builds.
I’m sort of surprised nobody’s mentioned Java. Isn’t that still popular for building this kind of thing? I haven’t used it to build a web site in over 10 years, so I’m not seriously suggesting it.