I am keen to know developers’ thoughts on how to make it easy for users to spin up and access a remote instance of Stan.
My assumption is that the remote code would be CmdStan or a derivative and that we’d replace the interface provided by, for example, CmdStanR, to a local instance of CmdStan, such that we had an R interface to a remote instance of CmdStan.
There are clearly issues around how to achieve the speed-up. However, the question here is, assuming that we could get the speed-up, how would we make that accessible to the users. One option would be some kind of API interface. I’m keen to know what other options people would recommend we consider.
not sure what you mean by access. inasmuch as a remote server is remote, communication will depend on your bandwidth; both compute cycles and bandwidth are expensive.
also not sure what you mean by “get the speed-up”. the expensive operation in Stan is computing gradients. the eco-groovy speed-up, is to compute smarter, not faster.
You are right that simply throwing compute at a problem is a bad plan and that working smarter is better than working harder.
Sorry if this wasn’t clear but the setting i am envisioning involves a Stan program that takes n hours to compute locally but where the user needs the answer in less than n hours. Assuming that the remote machine could be larger and that Stan could be configured to exploit the compute, the question here is how we best facilitate an interface that makes the user almost unaware of the fact that the algortihmic task was running remotely. That clearer?
I’ve been exploring this. I wanted to use PyStan but use Jupyter Notebooks a lot, I had an interest in learning async, and the whirring of my work laptop every time I started sampling was annoying me, so I started writing an async client that connects to a remote httpstan instance.
It kind of works with the httpstan server running on AWS, but I had a bunch of issues with the mimetype of the responses and haven’t had a chance to debug it in between actually doing the work, so I’m relying on noise cancelling headphones as my poor laptop does its best.
Eventually I’d like to set it up with boto3 so that you can just provide AWS credentials and it will set up a remote httpstan server (possibly just spinning up an ECS instance and doing all of the relevant wiring) and run models on that, but I need to get it working a little better first.
The biggest issue I have at the moment is that the response format from httpstan (newline-delimited JSON objects) is really inefficient, so I’ll likely look at trying to fork httpstan to preprocess the outputs and then look at some way to deliver just the pieces that are required before I get to that point.