A bug with elementwise pow in the 2.24 release

Unfortunately, we have a bug in the 2.24 that will require a 2.24.1 release: https://github.com/stan-dev/stanc3/issues/661

The issue only affects the use of elementwise pow so no urgent need for users to upgrade if you do not wish to use elementwise pow.

In the interest of improving our process, this fell through for the following reasons:

  • the reviewer (me) did not test whether the model compiles locally; I hate that we rely on local testing, but I should have remembered this - I apologize.
  • insufficient testing of new models that reside in stanc3

The PR in stanc3 that implemented this feature had all the proper test models, but the stanc3 tests themselves do not test whether the test models compile at the C++ level.

The tests we currently have in place to check whether models compile to the executables are:

There is an issue open that will improve this testing: https://github.com/stan-dev/stanc3/issues/636
Seems that we will need to prioritize this a bit more.

If there is any good news here that will only require a simple stanc3 PR, so we will only need to update the stanc3 binaries from the 2.24.0 release tarballs. The rest of the code remains the same.

5 Likes

This bug is related to a previous discussion we had in Using std::pow inside stan::math namespace

Yikes. Are we no longer testing that the good models compile? We used to require every signature got tested in a model instantiation that went all the way through compilation.

I don’t know anything about the continuous integration (other than that I couldn’t live without it), but it might not be that hard to add this functionality back again.

2 Likes

We do, the integration tests were actually expanded to multiple OS just a few weeks ago. However, these tests are run in the Stan repo, which means that the models are not updated with stanc3. Huge yikes indeed.

Which is why we need to make stanc3 a submodule of Stan so the tests models will be taken from stanc3 and also add upstream Stan tests to stanc3 (see linked issue above). And most importantly that we have linear history in Stan to see if and when things broke/regressed.

We realized this deffiency in our tests awhile ago (most of the changes proposed in https://github.com/stan-dev/design-docs/pull/22 are in this direction) and started the long process of refactoring the CI it but this has a bunch of steps, and making sure things work for all configurations across the multiple Jenkins agents takes its time. Luckily we have Nic to sort that out. We are slowly getting there, a big step was done just a few weeks ago where we now finnaly use stanc3 in Stan tests.

2 Likes

Before we do that we can add an ad-hoc test in stanc3 that just compiles with cmdstan(that is ready in stanc3) Its not the proper way of doing it (cmdstan is not the only consumer of stanc3) but it would help us in the meantime.

We need the long winded approach in the longer term because its not uncommon for a change to require orchestrating stuff across multiple repos and to facilitate that we need to do do this properly.

Just to make sure - does pow(real, real) (ie., the simple case of two scalars) work without issues in cmdstan 2.24?

Yes.

1 Like

I see a CmdStan 2.24.1 but no new release for stanc3.

cc @serban-nicusor please create a 2.24.1 release of stanc3 when you have a few minutes. Thanks!

1 Like

v2.24.1 for stanc3 is out and cmdstan-2.24.1.tar.gz updated with the new binaries.
Please let me know if everything is working fine now, thanks!

4 Likes