The issue only affects the use of elementwise pow so no urgent need for users to upgrade if you do not wish to use elementwise pow.
In the interest of improving our process, this fell through for the following reasons:
the reviewer (me) did not test whether the model compiles locally; I hate that we rely on local testing, but I should have remembered this - I apologize.
insufficient testing of new models that reside in stanc3
The PR in stanc3 that implemented this feature had all the proper test models, but the stanc3 tests themselves do not test whether the test models compile at the C++ level.
The tests we currently have in place to check whether models compile to the executables are:
in the stan repository we run integration tests that compile the so-called good models. These models are leftover from the stanc2 development times and do not have the new features.
If there is any good news here that will only require a simple stanc3 PR, so we will only need to update the stanc3 binaries from the 2.24.0 release tarballs. The rest of the code remains the same.
Yikes. Are we no longer testing that the good models compile? We used to require every signature got tested in a model instantiation that went all the way through compilation.
I don’t know anything about the continuous integration (other than that I couldn’t live without it), but it might not be that hard to add this functionality back again.
We do, the integration tests were actually expanded to multiple OS just a few weeks ago. However, these tests are run in the Stan repo, which means that the models are not updated with stanc3. Huge yikes indeed.
Which is why we need to make stanc3 a submodule of Stan so the tests models will be taken from stanc3 and also add upstream Stan tests to stanc3 (see linked issue above). And most importantly that we have linear history in Stan to see if and when things broke/regressed.
We realized this deffiency in our tests awhile ago (most of the changes proposed in https://github.com/stan-dev/design-docs/pull/22 are in this direction) and started the long process of refactoring the CI it but this has a bunch of steps, and making sure things work for all configurations across the multiple Jenkins agents takes its time. Luckily we have Nic to sort that out. We are slowly getting there, a big step was done just a few weeks ago where we now finnaly use stanc3 in Stan tests.
Before we do that we can add an ad-hoc test in stanc3 that just compiles with cmdstan(that is ready in stanc3) Its not the proper way of doing it (cmdstan is not the only consumer of stanc3) but it would help us in the meantime.
We need the long winded approach in the longer term because its not uncommon for a change to require orchestrating stuff across multiple repos and to facilitate that we need to do do this properly.