Prototype: Simplified CLI to Stan

[maedoc]

Just wanted to +1 the new format and add that it would likely be easier to do command line argument completion (as is common in cli tools in bash) for this format compared to the existing format.

[syclik]

@rok_cesnovar

What if we work on a fork of Cmdstan on someone’s account and slowly build stuff there with CI and everything. I think that would be fine in terms of collaboration.

That makes sense in the short term. I’ll set one up and post back when it’s done.

@rok_cesnovar

I am a bit worried that we will confuse users with another name and another interface. We already have a confusing structure of repos in my opinion and users have a hard time figuring out where to report errors or post requests.

Yes… but we’ll also confuse users if all their old scripts break. Especially since there’s no way to check for version using CmdStan just by calling the executable. (There’s no way to run something like ./exe --version or even ./exe version to check that first)

@rok_cesnovar

But you if want this to be a new project then I am not going to oppose, given that you did all the work.

What I’d want is what’s easiest for the developers and users. I think whatever we decide for developers is fine; right now it’s just me, but for those that work on CmdStan, I think we can easily deal with any of the reasonable options (same project, different project). For users, I think we should think through and plan. This includes CmdStanR/Py. Maybe it’s not such a big deal.

@rok_cesnovar

Re: logging levels

I may be wrong but wouldnt that mostly be changes to the services (stan repo).

One immediate thing is that we can turn off output completely. (Andrew asks for that in RStan all the time.)

@jonah

In terms of whether this should be CmdStan or something new: is there any reason to have both?

=). Nothing other than backwards compatibility. This prototype is not backwards compatible.

@mitzimorris

what all is deprecated? the spreadsheet on argument names is here: CmdStan sampler arguments - Google Sheets

Can I comment at will on that doc?

@mitzimorris

@rok_cesnovar

We could make a translator of the old commands to the new ones fairly easy.

agreed - this should be easy - are you thinking of something like a quick pre-scan of the args?

Funny enough, i went the other direction in order to create the same output files.

We could use the actual argument structure and change the print method to make that happen, I think. So… another executable.

@mitzimorris

I just revisited that document and the discussion…

+1 to that whole paragraph! I really like that proposal. This prototype doesn’t get all the way there and it’d isn’t simple to make it do that.

@maedoc

Just wanted to +1 the new format and add that it would likely be easier to do command line argument completion (as is common in cli tools in bash) for this format compared to the existing format.

Thanks! Argument completion would be rad. Any other feedback welcome too.

[syclik]

I put the prototype here:

generable/cmdstan

The README.md and description have been updated to indicate that it’s the simplified prototype. For practical purposes, it’s a fork of stan-dev/cmdstan so that when we’re happy with it, we can submit a PR back into stan-dev/cmdstan easily. (If it lives as a separate project, we’ll have to do some fancy git diff to do a reasonable PR.) Also, last I remember, you can’t fork your own repo into your own organization, so we can’t just set up a stan-dev/cmdstan-next that maintains the history; it’d have to be a completely new project.

Issues are enabled, so we can handle issues, pull requests, and code reviews.

Please feel free to contribute anything. Feedback, code, documentation, etc.

[jonah]

@syclik

Also, last I remember, you can’t fork your own repo into your own organization, so we can’t just set up a stan-dev/cmdstan-next that maintains the history; it’d have to be a completely new project.

Yeah I think it’s still not possible to have two forks within the same organization. But it’s possible to “duplicate” a repository without forking it, although I’ve never tried it:

Duplicating a repository - GitHub Docs

[rok_cesnovar]

@syclik

There’s no way to run something like ./exe --version or even ./exe version to check that first

This is actually in the works and was ready for the last release but there was not much interest in it. Add model compile information to stdout and csv by rok-cesnovar · Pull Request #896 · stan-dev/cmdstan · GitHub is approved but needs Add virtual function model_compile_info to the model_base.hpp by rok-cesnovar · Pull Request #2932 · stan-dev/stan · GitHub approved. Then another PR and

./examples/bernoulli/bernoulli compile-info (other ideas for name welcome)

would output:

MODEL_NAME = bernoulli_model
STAN_VERSION = 2.24.1
STANCFLAGS = --allow_undefined
STANC_VERSION = stanc3 2.24.1
STAN_THREADS = true

For more see Add ./model compile_info · Issue #887 · stan-dev/cmdstan · GitHub

[billdenney]

As an interested user and observer here, I do have an opinion on backward compatibility. Breaking backward compatibility seems like a good idea here as it would make the system easier to use. But, as someone who occasionally has to go back to old projects where I need it to just work right now because someone needed a subtle tweak to the result yesterday, having the old interface stick around for a bit is very helpful.

My preference for naming would be:

Name the current, preferred interface cmdstan and name the old deprecated interface cmdstan2. That will encourage people to change, but it will allow simple backward compatibility when it is really necessary/helpful. I’d suggest not going toward cmdstan-next because when it becomes the old version cmdstan-next-next doesn’t roll off the tongue (or onto the keyboard). :)

[syclik]

@billdenney, thanks for the reply! (I missed the notification on this thread.)

I have a similar desire for backwards compatibility, so it’s great to hear your opinion. Mind if I ask a few questions? Just to work out some of the nuance:

• do you keep binaries around after you’re done with a project? (alternate: you keep the scripts in a repo and build whenever you need to run it.)
• how do you handle CmdStan? Is it through cloning the repo, git submodule, or a download?
• is CmdStan versioned in your scripts?

[billdenney]

@syclik I’m not the most advanced Stan user.

I keep a Docker container with my Stan installation to be able to reproduce the Stan installation exactly, and I download CmdStan. But mostly, I run via R. For running via R, I am in the process of transitioning to CmdStanR. My main benefit of CmdStanR is that I occasionally use the Torsten extension for PK/PD.

When I use it directly, the docker container version is stored in the script outputs. So, I know what was used.

[syclik]

Thanks!