# Prototype: Simplified CLI to Stan

**URL:** <https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696>\
**Category:** General\
**Tags:** cmdstan\
**Created:** [August 27, 2020, 8:44pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696 "2020-08-27T20:44:41Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 27, 2020, 8:44pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/1 "2020-08-27T20:44:41Z")

</div>

Hi all,

I’ve created a prototype command line interface for Stan that’s meant to be easier to use than CmdStan. If anyone is a CmdStan users and has some time to check it out, please leave some feedback here on discourse.

There are more details in this blogpost: [https://www.generable.com/blog/2020/08/prototype-simplified-cli-to-stan/](https://www.generable.com/blog/2020/08/prototype-simplified-cli-to-stan/)

For those that don’t want to click through, here’s some of the information.

* * *

### How to get the Prototype

The prototype is currently on a branch of CmdStan, [`feature/cli11`](https://github.com/stan-dev/cmdstan/tree/feature/cli11).

### Example: setting `delta` to 0.9 and `max_depth` to 14

Prototype:

```
> examples/bernoulli/bernoulli sample --delta 0.9 --max_depth 14 --data_file examples/bernoulli/bernoulli.data.R

```

CmdStan v2.24.0:

```
> examples/bernoulli/bernoulli sample adapt delta=0.9 algorithm=hmc engine=nuts max_depth=14 data file=examples/bernoulli/bernoulli.data.R

```

In the prototype, the order of the options doesn’t matter. In CmdStan, arguments are nested, so you need `algorithm=hmc engine=nuts max_depth=14` to set the max treedepth.

### Feedback

If you have any feedback, good or bad, please post it here or dm me.

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 27, 2020, 8:49pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/2 "2020-08-27T20:49:28Z")

</div>

cc: @mitzimorris, @rok_cesnovar, @wds15, @Bob_Carpenter, @stevebronder.

I’m trying to tag CmdStan devs and users. Please let them know this exists.

---

<div class="post-metadata">

**Author:** ![ahartikainen](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ahartikainen/32/321_2.png) [@ahartikainen](https://discourse.mc-stan.org/u/ahartikainen)\
**Post date:** [August 27, 2020, 8:51pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/3 "2020-08-27T20:51:38Z")

</div>

Looks clean.

Would `--delta=0.9` work?

This interface is assuming there are no overlap between the “nested” structure? (This is fine assumption in my opinion)

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 27, 2020, 8:55pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/4 "2020-08-27T20:55:41Z")

</div>

> [@ahartikainen](#):
>
> Would `--delta=0.9` work?

Yup.

> [@ahartikainen](#):
>
> This interface is assuming there are no overlap between the “nested” structure? (This is fine assumption in my opinion)

Not sure what this means… some clarification?

---

<div class="post-metadata">

**Author:** ![ahartikainen](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ahartikainen/32/321_2.png) [@ahartikainen](https://discourse.mc-stan.org/u/ahartikainen)\
**Post date:** [August 27, 2020, 8:57pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/5 "2020-08-27T20:57:19Z")

</div>

> [@syclik](#):
>
> Not sure what this means… some clarification?

Yes, I meant no overlap between the keywords/arguments.

Do we currently have any keywords/arguments which has same name but do different things?

---

<div class="post-metadata">

**Author:** ![jonah](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jonah/32/200_2.png) [@jonah](https://discourse.mc-stan.org/u/jonah)\
**Post date:** [August 27, 2020, 8:58pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/6 "2020-08-27T20:58:12Z")

</div>

> [@syclik](#):
>
> I’ve created a prototype command line interface for Stan that’s meant to be easier to use than CmdStan.

Cool. I definitely find CmdStan rather cumbersome to use, so anything that makes it simpler sounds intriguing! I’ll take a look more closely soon.

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 27, 2020, 9:00pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/7 "2020-08-27T21:00:28Z")

</div>

> [@ahartikainen](#):
>
> Yes, I meant no overlap between the keywords.

I think I understand.

Within a particular subcommand (e.g. `sample` vs `optimize`) the keywords can’t overlap. But that isn’t really a problem in our arguments.

I had considered doing something more substantial, but wanted to wait before doing anything with more effort. Something larger should really be done with a design in mind. This one, I just hacked together.

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 27, 2020, 9:04pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/8 "2020-08-27T21:04:18Z")

</div>

Maybe I should be a little clearer. I didn’t just throw it together with throw-away code. I think it’s clean and thought through well enough for what it is. It just wasn’t something that had an API design up front, software design, then implementation. All three happened organically. There are plenty of different ways to implement the same thing. There are some choices I made that other people would have made differently. That’s why this is a prototype… it’s something I put together (with minimal feedback).

I showed it to @mitzimorris and @RTrangucci a few days ago. But otherwise, it was just a solo effort.

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [August 27, 2020, 9:11pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/9 "2020-08-27T21:11:01Z")

</div>

> [@ahartikainen](#):
>
> Do we currently have any keywords/arguments which has same name but do different things?

yes, the keyword “file” is ambiguous between data\_file and output\_file.  
otherwise, the keywords are not ambiguous.

here’s a nice comparison of the library used to handle the command line arguments with the `boost::program_options` library: [Comparing CLI11 and Boost PO -](https://iscinumpy.gitlab.io/post/comparing-cli11-and-boostpo/)

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [August 28, 2020, 6:44am UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/10 "2020-08-28T06:44:37Z")

</div>

I like this a lot!

I would really like to see this in cmdstan, if possible, not as a separate repo. We will very likely have to bump major version for the incoming language changes in the not-so-distant future anyways. And there is now a substantial amount of deprecated stuff, etc. Maybe we can do this together with that? Another suggestion would be to add this to the cmdstan3 design doc ([https://github.com/stan-dev/design-docs/pull/15](https://github.com/stan-dev/design-docs/pull/15))?

We could make a translator of the old commands to the new ones fairly easy.

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 28, 2020, 2:42pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/11 "2020-08-28T14:42:55Z")

</div>

> [@rok\_cesnovar](#):
>
> I would really like to see this in cmdstan, if possible, not as a separate repo.

Awesome! If there’s enough demand, we could do that.

> [@rok\_cesnovar](#):
>
> We will very likely have to bump major version for the incoming language changes in the not-so-distant future anyways.

If we’re going to bump versions, I think there’s actually some substantial change we can make to make the arguments make more sense. Some things that I didn’t do in order to be compatible with CmdStan:

- rename `nun_samples` to `num_draws` or even `draws`
- change `data_file` to something like `input_file` or something similar so the short name is `-i` to match the output of `-o` (like other programs)
- move fixed param out of sample. It’s not really sampling. It’s closer to generate quantities in some sense
- add much more to diagnostics. @rok_cesnovar, benchmarking?
- properly handle logging levels
- different input and output formats

There’s a lot more that could be done.

> [@rok\_cesnovar](#):
>
> Another suggestion would be to add this to the cmdstan3 design doc

This actually doesn’t address those design points. I thought about it and didn’t go that direction even though I wanted to. To do it properly from c++, we need to runtime load libraries, which there isn’t cross platform compatible. So… a bunch of headache there. We could do this using something like a python or bash script… but that’s something different too and wouldn’t have fixed the command arguments.

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [August 28, 2020, 3:04pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/12 "2020-08-28T15:04:46Z")

</div>

> [@syclik](#):
>
> I think there’s actually some substantial change we can make to make the arguments make more sense.

@jonah and @mitzimorris have done a bunch of work on argument names. The link to a google sheet should be somwhere in the cmdstan3 design doc somewhere. I think some of your suggestions are already in there.

> [@syclik](#):
>
> If we’re going to bump versions,

I think we have been circling around this long enough we have come to the point we should just decide on it and do it. Doing it for the next release cycle (October) is too soon, but the release after that is probably doable. But we need to set up a plan, prepare a list of things we need to deprecate, etc

And yes, a bump should have a bunch more stuff. We do not want to bump major versions just so we can remove deprecated stuff.

---

<div class="post-metadata">

**Author:** ![ahartikainen](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ahartikainen/32/321_2.png) [@ahartikainen](https://discourse.mc-stan.org/u/ahartikainen)\
**Post date:** [August 28, 2020, 3:28pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/13 "2020-08-28T15:28:03Z")

</div>

Is there `--help` ?

Answer: of course there was

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 28, 2020, 4:15pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/14 "2020-08-28T16:15:34Z")

</div>

I could also add a `--help-all` and I did at some point. But didn’t find it too useful. Too much going on.

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [August 28, 2020, 4:19pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/15 "2020-08-28T16:19:56Z")

</div>

> [@rok\_cesnovar](#):
>
> But we need to set up a plan, prepare a list of things we need to deprecate, etc

Yes, all the logistics of making a non-breaking change.

I’m sort of leaning towards calling it something else so the logistics are a little easier. Cause now that it exists, I can already make things better like changing logging levels.

If we’re looking at a 6 month timeframe, it also makes sense to incubate it as a different project until it’s set. Meaning… CmdStan is mature and not going to change much. This prototype could change between now and when it gets a proper release. And that’s fine and expected. Hard to be collaborative on a single branch. (GitHub isn’t really designed around that.)

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [August 28, 2020, 4:40pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/16 "2020-08-28T16:40:05Z")

</div>

What if we work on a fork of Cmdstan on someone’s account and slowly build stuff there with CI and everything. I think that would be fine in terms of collaboration.

I am a bit worried that we will confuse users with another name and another interface. We already have a confusing structure of repos in my opinion and users have a hard time figuring out where to report errors or post requests.

But you if want this to be a new project then I am not going to oppose, given that you did all the work.

Re: logging levels

I may be wrong but wouldnt that mostly be changes to the services (stan repo).

---

<div class="post-metadata">

**Author:** ![jonah](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jonah/32/200_2.png) [@jonah](https://discourse.mc-stan.org/u/jonah)\
**Post date:** [August 28, 2020, 4:58pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/17 "2020-08-28T16:58:58Z")

</div>

In terms of whether this should be CmdStan or something new: is there any reason to have both? My initial thought is that nobody really benefits from having both (neither users nor developers), so we should just go with the better one. If this new one is preferred then that means we either call it CmdStan or we call it something else and retire the name CmdStan.  
Anyway, that’s my initial inclination, but I could certainly be overlooking reasons to maintain both. Does anyone think we’d be better off maintaining both separately?

---

<div class="post-metadata">

**Author:** ![ahartikainen](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ahartikainen/32/321_2.png) [@ahartikainen](https://discourse.mc-stan.org/u/ahartikainen)\
**Post date:** [August 28, 2020, 6:02pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/18 "2020-08-28T18:02:00Z")

</div>

If API and parts are going to change much, then something similar as in `pystan-next` could be possible, but if most of the code is same, I’m not sure if there is a better way to do this than a new branch?

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [August 28, 2020, 6:30pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/19 "2020-08-28T18:30:35Z")

</div>

> [@rok\_cesnovar](#):
>
> I would really like to see this in cmdstan, if possible, not as a separate repo. We will very likely have to bump major version for the incoming language changes in the not-so-distant future anyways. And there is now a substantial amount of deprecated stuff, etc.

what all is deprecated? the spreadsheet on argument names is here: [CmdStan sampler arguments - Google Sheets](https://docs.google.com/spreadsheets/d/1lU6vcY3GF_ftlFdCe2kDhr5FC4YJPWauC5bp2D9zz74/edit#gid=0)

there are a few arguments which are candidates for deprecation - are there other items?

> [@rok\_cesnovar](#):
>
> We could make a translator of the old commands to the new ones fairly easy.

agreed - this should be easy - are you thinking of something like a quick pre-scan of the args?

> [@rok\_cesnovar](#):
>
> Another suggestion would be to add this to the cmdstan3 design doc ([https://github.com/stan-dev/design-docs/pull/15](https://github.com/stan-dev/design-docs/pull/15))?

I just revisited that document and the discussion. as Daniel said, we’d need to figure out how to link in the compiled models to make the proposed syntax work. the ambition level of design doc proposal falls between the amount of workflow management provided by the R and Python interfaces and the current CmdStan interface, and maybe that’s not really that useful - note that under that proposal CmdStan still only runs a single chain - it assumes that the user is comfortable working in a \*nix or DOS terminal window and is also OK writing simple shell scripts in order to run multiple chains and manage the output files accordingly - (cf [4 MCMC Sampling | CmdStan User’s Guide](https://mc-stan.org/docs/2_24/cmdstan-guide/mcmc-intro.html#running-multiple-chains)).

I’m all for this new way of handling arguments - a sub-argument “method” followed by any number of arguments in any order.

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [August 28, 2020, 6:43pm UTC](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696/20 "2020-08-28T18:43:55Z")

</div>

> [@mitzimorris](#):
>
> there are a few arguments which are candidates for deprecation - are there other items?

There are a bunch of functions to be deprecated, the ← assignment operator, # comment character,bin/print and so on. I was refering to that.

[Next page](https://discourse.mc-stan.org/t/prototype-simplified-cli-to-stan/17696.md?page=2)