New array syntax might mean Stan language 3.0?

[Bob_Carpenter]

@andrewgelman

Every time I code one of these models, flame out, and get rescued by the Stan team, I learn so much! Thanks again.

You’re welcome!

The question hanging over all of this is how we can make it easier for statisticians to write robust and efficient models. Ideally, a Stan user wouldn’t need a dedicated statistician, numerical analyst, and programmer to get their domain model off the ground. But I don’t see how to eliminate any of those roles in state-of-the-art models, which run into trouble on all three fronts pretty quickly.

[andrewgelman]

Yes, I struggle with this too. And this particular model was not very complicated. I’m hoping that our forthcoming Bayesian Workflow in Stan book will help.

[seantalts]

@Bob_Carpenter

Doing this will break all the case studies, books, etc. that use arrays, which means pretty much everything.

I think I was accidentally somewhat misleading here - @rybern’s PR accepts both old and new array syntax (though not sure how this meshes with tuples) and just breaks programs that use array as a variable name.

@Bob_Carpenter

As @rok_cesnovar points out, we’ve been pretty slack on the reserved words. Every time we introduce a function we don’t allow variables to be named that any more.

Given this and the point above, would you be in favor of releasing this change as part of Stan language 2.0 still and avoiding the major version bump? I am somewhat confused by the stance in general here - I see a github issue for e.g. un-reserving multiplier and offset and that we’re now allowing variable names that match math function names, so that side seems to aim towards avoiding adding new reserved words in the same major version if possible (maybe case-by-case?). I think in general I am in favor of keeping our number of reserved words small - large preference for not reserving math function names (check!), smaller preference for not reserving the word array if possible.

For the purposes of discussion, I’ll propose that @rybern and I (and @Matthijs if he’s around) try to dig into what we’d need to do to avoid reserving the array keyword (possibly un-reserving other type and constraint names in the process if desired). If that’s too hairy, then maybe we do an option @rok proposed earlier, which is to merge all of the backwards-compatibility-breaking changes into a stan-3.0 branch (with conversion tool) until we’ve built up a good number of them and can have a 3.0 release worthy of a party, haha. What do you all think?

[Bob_Carpenter]

@seantalts

and just breaks programs that use array as a variable name.

I’m OK doing this as we’ve done this for every other function and language keyword we’ve introduced.

@seantalts

I am somewhat confused by the stance in general here -

Not surprising given that we have never strictly followed semantic numbering when it came to reserved words. We add reserved words whenever we add a new function (e.g., reverse) or data type (e.g., cholesky_factor) or control keyword (e.g., reject). If we had been following semantic numbering strictly we’d be on something like version 75.0.0 of the language.

@seantalts

I’ll propose that @rybern and I (and @Matthijs if he’s around) try to dig into what we’d need to do to avoid reserving the array keyword (possibly un-reserving other type and constraint names in the process if desired).

I agree that’s a better solution.

@seantalts

which is to merge all of the backwards-compatibility-breaking changes into a stan-3.0 branch (with conversion tool) until we’ve built up a good number of them and can have a 3.0 release worthy of a party, haha. What do you all think?

I can’t think of any reason to wait longer on this. It’ll break some old models laying around that still use <- for example, but otherwise should be fine. In all of these cases there’s a simple direct translation to 3.0 given in the deprecation appendix of the reference manual.