I like the current release cycle. In Hotfix release thread Hotfix release I suggested that in future it would be nice to have release candidates. I also briefly talked about this in Thursday Stan meeting. I write here a bit more details.
- Continue as now with announcing feature freeze time for developers
- After the feature freeze starts, prepare quickly release notes which tell what new features will be in the next release
- Prepare release as usual, but before tagging it as release, tag it as a release candidate. Release candidate should be in state that we assume it should be as easy to install as a regular release.
- Advertise release candidate and the new features for users, and ask them to test installing and running some basic model and one of their favorite models.
- If there are show-stopping issues, fix them and make a new release-candidate.
- If there are no show-stopping issues in couple days, make a release.
The benefit of release candidate is that we get installation and usage tested in larger variety of environments and with larger variety of models. Those who need more stability can wait for the actual release. If there are show-stoppers, there is not as big panic for making hotfix. The cost is couple extra days to wait that users have time to test, but if all is right then all is needed is to rename the tag for the actual release.
This proposal is specifically for math/stanc/stan/cmdstan. With help from CmdStanR and CmdStanPy, installing a release candidate can be made easy without breaking users’ other installations.
For example, with CmdStanR (by adding a new repo_tag argument)
RCPATH = "~/.cmdstanr/cmdstan-rc"
dir.create(RCPATH)
install_cmdstan(dir=RCPATH, repo_tag="v2.23.rc1")
set_cmdstan_path(RCPATH)
testmodel = cmdstan_model("testmodel.stan", quiet = FALSE)
fit = testmodel$sample(num_warmup = 400,
num_samples = 400,
num_chains = 4,
num_cores = 4)
## when you are ready with the testing you can remove the release candidate
unlink(RCPATH, recursive=TRUE)
This and similar instruction for CmdStanPy could be included in the release candidate announcement.
There could be additional examples to test new features which may need additional compiler options (which can be given to cmdstan_model() like new GPU and MPI features.
EDIT: added dir.create