Just wanted to give a heads-up on the plan for the next release - version 2.32
The proposed plan is:
- the feature freeze on the 4th of April,
- the official release on the 17th of April.
Please let me know if you think any of these dates should be moved. We are pretty flexible this time around, barring any surprises, this should be a pretty minor release.
If you need a pull request reviewed or think a bug fix is a high priority for this release, please use this thread to report it.
Just want to double check: is 2.32 going ahead with removal of the deprecated array syntax? If so, it’d be worth thinking of this as a major release in terms of its impact on the community. e.g.
I’m working on the PR which will formalize this now, but yes, that is currently the plan. For 2.32, the expiring deprecations will become errors, but will still be parseable and
--canonicalize=deprecations will recognize them and produce the updated syntax.
For 2.33 and on, they will not be recognized at all. It’s looking like Tuples will finally be able to land in 2.33, and they are really reliant on the new array syntax, so a delay in the deprecation schedule could start holding up features. If there are things we can do to make this less painful for the R packages I’d love to hear them
Tagging @paul.buerkner, as
brms is by far the most prominent R package that depends on running Stan code via both
cmdstan and CRAN’s
rstan. Edit: the only strict dependency is on CRAN’s rstan, but running via cmdstan is important functionality.
If Paul feels good about the way forward, then I’m happy.
Is it planned that the auto-formatter will be able to digest the old Stan code with 2.33 and onwards?
I’d guess that as long as all this means is to switch on some parser option, then it’s manageable. In case the old syntax cannot be parsed at all with 2.33 going forward, then that could be an issue, I guess.
(great to read implicitly that tuples are going to land in 2023 presumably)
The plan is that in 2.33, the compiler can forget about the old syntax entirely. If the goal of deprecations is to allow the language and compiler code base to evolve, freezing the auto formatter would prevent us from actually deleting that old code.
Of course, if you have code from 2.19 or whatever version before 2.33, you can run the auto formatter from an “old” version (e.g. 2.31) as part of your update cycle and that will always work. It might be reasonable to provide a really striped down binary which only does migration of pre-2.33 code (e.g. not even compilation), since that binary would never really need to be updated.
For things I would like to see merged before the release:
Hey @spinkney, this sounds good! Is there anything else to do for me that helps finishing this PR until April?
I predict we get a lot of problems if RStan in CRAN is not supporting new array syntax when the old syntax stops working in a new release. It would be good to be certain that there is a new working CRAN RStan before removing the old array support. Ping @bgoodri @andrjohns
I don’t think so. I know @andrjohns is reviewing your PR but he’s swamped with Rstan cran stuff.
Before the freeze we need to either merge the PR which removes the deprecations or put together a new one which moves the removal versions back, so if we are deviating from the deprecation schedule it needs to be an active choice made soon. I’m not sure what the status of CRAN submission is, last I heard it was close (@andrjohns @hsbadr)
I believe 2.26 should be ready and there is only one package which will additionally fail under 2.31 (but they have a patch submitted which should be out to CRAN soon).
But @bgoodri is handling submission, so will let him clarify
I’ve also consolidated the current status of
rstan downstream package failures/patches here: RStan & StanHeaders Downstream Patching Status · Issue #1053 · stan-dev/rstan · GitHub
There are now no additional failures under 2.31, so once
StanHeaders 2.26 and
rstan 2.26 are on CRAN, the upgrade process should be painless.
Given this (assuming a CRAN submission for 2.26 soon), one option could be to push the deprecation removals back one more version. Then
rstan users will have roughly a release cycle to prepare - and we’ll have the same to prepare downstream packages.
On that note, would it be difficult to build a
stanc.js where the deprecations are errors? Then I can run the reverse-dependency checks and see what patches are needed
EDIT: Never mind about the
stanc.js, I didn’t think to just pull it from the PR
I recently did a check of our own CI to make sure none of the Stan or CmdStan test models would fail, so I have a Jenkins run with artifacts here: Artifacts of stanc3-test-binaries #25 : /bin [Jenkins]
Looks like the downstream failures are pretty extensive with the deprecations-as-errors, but I’ve suggested using the auto-formatter during the package installation as an interim solution: Auto-format Stan models during rstan_config()? · Issue #113 · stan-dev/rstantools · GitHub
That will only work for 1 version post-removal, so it’s not the best long-term solution. At some point the .stan files themselves will need to be updated on disk, not just during parsing.
Is it necessary to remove that functionality from the auto-formatter? It could be useful for users in the same way that it has been for deprecated syntax.
Not that I’m advocating for this as a long-term solution in any way - I wouldn’t want it to be in place for more than a version anyway, otherwise we’re promoting/enabling incorrect syntax
If the point of the deprecation policy was to improve the codebase, then yes it is necessary. The code the compiler is required to carry around to parse the old syntax is the same whether we’re auto-formatting it or compiling it down to C++, so we can’t actually move on until it is really gone.
We’re not going to be intentionally mean about this, so things which are “cheap” to update (e.g., turning multiply_log into lmultiply) might stick around longer or even forever, but we only guarantee it for one version.
If you know that the source code you’re dealing with is 100% old syntax, there’s no reason you couldn’t use a 2.31/2.32 compiler to do the update forever into the future, but if you’re not careful with enforcing the deprecation policy it’s easy to imagine a year from now someone will try to write code which uses e.g. a new function from 2.33 with the old array syntax, and suddenly there’s no compiler that exists which can handle both those things.