# Stan Math Release notes draft

**URL:** <https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121>\
**Category:** Developers\
**Tags:** math\
**Created:** [September 25, 2019, 5:58pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121 "2019-09-25T17:58:36Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [September 25, 2019, 5:58pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/1 "2019-09-25T17:58:36Z")

</div>

There was a debate on one of the closed Stan Math PRs regarding the release notes. I didnt feel like having another long winded discussion so I took 15 minutes and did a first draft.

Two questions:

- Does anyone mind if we open a new wiki page where we keep an updated list of such one-liners? If you feel helping out this process just add a one-liner “Release notes” to each Math PR you open. If not, that is also fine with me, dont want to set new rules just for this. Its only 15 minutes/3 months :).
- Any objections to the draft?

* * *

Stan-user facing changes:

- added a new function to solve systems of nonlinear algebraic equations using Newton-Krylov methods.
- re-implemented a reentrant safe and scalable version of lgamma
- change the default boost\_policy\_t to avoid the double to long double propagation, this could affect the following functions: falling\_factorial, rising\_factorial, digamma and lgamma
- implemented oridinal\_logistic\_glm\_lpmf
- fixed boundary conditions for log\_sum\_exp and log\_diff\_exp
- GPU support for the following functions: gp\_exp\_quad\_cov, mdivide\_right\_tri, multiply, mdivide\_left\_tri, normal\_id\_lgm, bernoulli\_logit\_glm\_lpmf and poisson\_log\_glm\_lpmf

Testing:

- added a new autodiff testing framework for simpler and more automated tests
- added finite difference versions of gradient, Hessian, and gradient of Hessian functions that automatically scale step size based on input
- cleaned up test behaviour when -NDEBUG is set
- tests for bernoulli\_logit, poisson\_log and negbin\_2\_log GLMs are now tested with y as std::vector instead of Eigen::Matrix for better test coverage of the usual use case from Stan

Refactor/Code cleanup/Technical debt:

- refactor of the type traits to use more C++14 features & SFINAE
- cleaned up rev/mat/fun with the eigen plugin methods .val(), .adj() and .vi()
- typename \*\_return type was changed to \*\_return\_type\_t
- removed deprecated Eigen content from math headers
- metaprograms cleanups: added various enable\_if\_\*
- added use of const ref and ref to replace copies with moves
- added clang-tidy to the makefiles
- cleaned up unnecesary Boost and Gtest compiler flags
- beta binomials lpmf loop and conditionals cleanup
- clang-tidy cleanup of the Stan Math codebase (adding braces, replace typedef with using and NULL with nullptr)
- the make test-math-dependencies file was rewritten from make to Python

OpenCL/GPU:

- GPU support for prim/gp\_exp\_quad\_cov, prim/mdivide\_right\_tri, prim/multiply, rev/multiply
- fixed a bug with padding matrices
- added the triangularity view to matrix\_cl
- the key for opencl kernel options is now a std::string not a const char \*
- added move constructors and assigments to avoid unnecesary copying
- all references to “GPU” were changed to “CL”
- changed the copy function to to\_matrix\_cl and from\_matrix\_cl
- updated the OpenCL headers which also allowed for some OpenCL core code cleanup

---

<div class="post-metadata">

**Author:** ![stevebronder](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stevebronder/32/17242_2.png) [@stevebronder](https://discourse.mc-stan.org/u/stevebronder)\
**Post date:** [September 25, 2019, 6:18pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/2 "2019-09-25T18:18:25Z")

</div>

Thank you Rok!! It would be nice to have the PR # and person who submitted so we can give contributors props. That’s a bit tedious, tmrw I’ll see if I can write a script that grabs info for all the PRs merged since the last release. @syclik do you have something like that already?

Do we have any speedup graphs for the OpenCL code? I can write up a draft tomorrow for the more narrative release notes (unless Dan or Bob wants to take the wheel on that of course)

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [September 25, 2019, 7:22pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/3 "2019-09-25T19:22:27Z")

</div>

I can add PR numbers and names (github handles), thats probably a 5-10min job anyways. I will send you the slides with the speedups. Thanks!

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [September 26, 2019, 12:52pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/4 "2019-09-26T12:52:40Z")

</div>

This is great!

I have no objections. I used to do it manually and it was more like 2 hours every release, but if you can do it faster, great!

Steve, nope. I did it manually using filters on the issues. Go for it!

Rok, you’re missing a couple user-facing things:

- the lgamma PR changed the default compiler options; all interfaces should be updated.

- numeric computations with the last version might have changed if it called boost; this was due to overriding the default boost policy with a custom one that’s changed this release. So this version may not replicate the last version.

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [September 26, 2019, 3:44pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/5 "2019-09-26T15:44:52Z")

</div>

Thanks, Rok.

> [@stevebronder](#):
>
> It would be nice to have the PR # and person who submitted so we can give contributors props.

We used to try to link the commits at one point and give props to bug reporters and anyone not on the stan-dev list.

Automating all this sounds great, but until then, great on Rok for doing it manually! I don’t think anyone will object.

> [@syclik](#):
>
> I used to do it manually

It started with Daniel and I doing all the triage and writing all the release notes by scrolling through the commit messages, issues, and PRs. That was a terrible process and we missed a lot of stuff.

> [@rok\_cesnovar](#):
>
> There was a debate on one of the closed Stan Math PRs regarding the release notes.

At the risk of starting a debate, I wouldn’t have even called that a debate! The only issue in that discussion that touched release notes was whether they could be automated by including a special field in PRs.

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [September 27, 2019, 2:58am UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/6 "2019-09-27T02:58:15Z")

</div>

I still think it’s good to have the end Stan user in mind when creating, editing, working on, and releasing issues in the Math library. Adding a release notes block that is to be written from the Stan user’s perspective would suffice.

---

<div class="post-metadata">

**Author:** ![syclik](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/syclik/32/6_2.png) [@syclik](https://discourse.mc-stan.org/u/syclik)\
**Post date:** [September 27, 2019, 4:27am UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/7 "2019-09-27T04:27:45Z")

</div>

> [@seantalts](#):
>
> I still think it’s good to have the end Stan user in mind when creating, editing, working on, and releasing issues in the Math library.

Do you mean “creating releases” instead of “releasing issues”?

I also think it’s good to think about and write for the Stan user, but it’s an awkward policy to force issues in Math to be written that way. If that’s what we all agree on, that’s fine, but think about how this would affect all the other parts of the Stan ecosystem. Do you think it’s going to be productive to write stanc3 issues so that it’s written for the Stan user?

I’m ok if we decide to adopt a general policy to state how an issue affects the Stan user (this may be left blank if it’s an internal change). One reason to not do this is because it adds yet another requirement on the person adding the issue / PR (wherever the policy is applied). There’s also an issue that the person writing the issue or code may not have the knowledge to write this down. In that case, what happens? Is the issue supposed to be rejected?

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [September 27, 2019, 10:15am UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/8 "2019-09-27T10:15:38Z")

</div>

> [@syclik](#):
>
> I used to do it manually and it was more like 2 hours every release, but if you can do it faster, great!

There were a bit over 50 PRs open since the last release, some of them were closed before merge. Most of the merged once I was either the author or the reviewer or they had a large enough effect on my work that I was aware of them. I only had to actually click on maybe 5 PRs. So I had it easy this time.

> [@syclik](#):
>
> Rok, you’re missing a couple user-facing things:

Thanks for those. Also added the Newton solver added yesterday. Will keep the top-level comment updated until the release.

> [@Bob\_Carpenter](#):
>
> At the risk of starting a debate, I wouldn’t have even called that a debate! The only issue in that discussion that touched release notes was whether they could be automated by including a special field in PRs.

That is fair. I just felt we circled around this in various places in the last few months.

---

<div class="post-metadata">

**Author:** ![stevebronder](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stevebronder/32/17242_2.png) [@stevebronder](https://discourse.mc-stan.org/u/stevebronder)\
**Post date:** [October 16, 2019, 4:30pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/9 "2019-10-16T16:30:16Z")

</div>

Alrighty since we have the feature freeze I went through and wrote up an R script to grab all the merged PRs and wrote up a few release notes for this cycle. The gist below has both the R script to grab the PR info along with a little narrative for the changes this release.

I tried to tag all parties and mention their contributions. If anyone would like to change the wording lmk!

> <https://gist.github.com/SteveBronder/8a3fe3b5ea1b71eb192f9e3053afa333>

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [October 16, 2019, 6:20pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/10 "2019-10-16T18:20:44Z")

</div>

Steve this looks fantastic! I will do a more detailed look tomorrow, the text up top is light out.

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [October 18, 2019, 7:02am UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/11 "2019-10-18T07:02:20Z")

</div>

Overall this is really really great. Thanks for doing it!

Here are my minor comments on the release notes.

> PR [#1180](https://github.com/stan-dev/math/pull/1180) brought the [Intel TBB](https://github.com/intel/tbb) into Stan as a dependency, which we will be using in the future for CPU parallelism all across Stan! The TBB is an excellent framework that will let Stan utilize nested parallelism across the algorithms and gradient evaluations. On top of that, it also supports OpenCL interop, so we will start experimenting with integrating out OpenCL backend with the TBB. The licensing for the Intel TBB library is under the Apache 2.0 license. This dependency implies an additional restriction as compared to the new BSD license alone. The Apache 2.0 license is incompatible with GPL-2 licensed code if the software if distributing the software as a unitary binary. Refer to the Apache 2.0 evaluation page on the Stan Math [wiki](https://github.com/stan-dev/math/wiki/Apache-2.0-License-Evaluation).

I would remove the “On top of that, it also supports OpenCL interop, so we will start experimenting with integrating out OpenCL backend with the TBB.”. I would focus on the features that are added as of now. And I feel that this steals the focus of TBB and may confuse users to connect TBB with OpenCL.

> With @t4c1’s large contributions, we now have GPU/OpenCL support for many of our glm functions! With these additions along with @rok-cesnovar’s work on integrating OpenCL optimizations in the new stan3 compiler should give users a pretty big speedup for large glm models! @rok-cesnovar added an OpenCL reverse mode specialization for `multiplication` and `mdivide_left_tri` while @t4c1 added the OpenCL specialization for `gp_exp_quad_cov` .

The second sentence is not valid anymore as 2.21 will be using the exisiting compiler, so I would just remove it. And the first sentence should somehow note that gpu support is only available inside Stan Math and not to the cmdstan users so people dont get the wrong expectations. They will be available immediately once we switch to stanc3.

> I (@Stevebronder) added a small type traits metaprogramming scheme so that we can make use of more generic templating in a lot of our code.

The sentence should start with “@Stevebronder added”. And the “small” her is not valid in my opinion. It was a huge effort and it helps developers a lot! Dont sell yourself short here!!

I would add a heading before the last paragraph with “Future”, “What is next?”, “A look at the next release”.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [October 18, 2019, 7:16am UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/12 "2019-10-18T07:16:09Z")

</div>

> [@rok\_cesnovar](#):
>
> I would add a heading before the last paragraph with “Future”, “What is next?”, “A look at the next release”.

Exactly!

When reading through the text - which is really nice overall - I also thought that we should only focus on what we are releasing and not what we plan to do with it. I think that we should not include things like what we plan to do with these things in release notes. These forward directed things can certainly be mentioned in an announcement of the release, but release notes are just a record of what’s in it; for me.

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [October 18, 2019, 1:09pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/13 "2019-10-18T13:09:16Z")

</div>

Hey all,

The release is today! Is there a canonical set of release notes we should include instead of the standard stuff for Math? I could also imagine release notes that are Math-dev specific (to be included in the Math repo RELEASE\_NOTES.txt file) and then some other ones that we might instead include higher up in Stan release notes that are more language-facing.

Just want to tag @serban-nicusor as the release-performer so he’s aware - I think these folks would like to write the release notes for Math this time.

Thanks everyone!

---

<div class="post-metadata">

**Author:** ![stevebronder](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stevebronder/32/17242_2.png) [@stevebronder](https://discourse.mc-stan.org/u/stevebronder)\
**Post date:** [October 18, 2019, 3:52pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/14 "2019-10-18T15:52:44Z")

</div>

Thanks for the corrections!! I fixed up the release notes so the forward looking stuff is removed (and the stuff about the new compiler :-/ )

@serban-nicusor @seantalts As long as everyone else is cool with the release notes above then I think we want to use these for the release!

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [October 18, 2019, 4:07pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/15 "2019-10-18T16:07:59Z")

</div>

Looks great :) Thanks Steve!

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [October 18, 2019, 6:01pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/16 "2019-10-18T18:01:57Z")

</div>

Looks good to me. @wds15 or @rok_cesnovar or @stevebronder (or?) - any of you want to write something up showing how much faster map\_rect with threading is now for the Stan release notes and pick a favorite graph to link to?

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [October 18, 2019, 6:22pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/17 "2019-10-18T18:22:02Z")

</div>

Would something like this work?

We are now using TBB for threading in map\_rect. With performance tests on a [non-trivial map\_rect model](https://github.com/syclik/benchmark-stan/blob/math/tbb/benchmark-warfarin.stan) we have observed speedups of up to 20% on Windows, 70% on Linux and 30% on MacOS. Speedups were observed for both Intel and AMD CPUs. On MacOS we observed 25-30% speedups even for single threaded models when using tbbmalloc.

We can remove the last line if you find it confusing for users. I feel that MacOS users could really benefit here by simply turning on `CXXFLAGS= -DSTAN_THREADS`. But that doesnt need to be a part of the releases.

As for the chart I like this one from Steve (I think its a 12-core AMD CPU):

[![](https://user-images.githubusercontent.com/5857231/66257806-3158c780-e7a6-11e9-9c8c-fbf6bcde8fd5.png) ](https://user-images.githubusercontent.com/5857231/66257806-3158c780-e7a6-11e9-9c8c-fbf6bcde8fd5.png)

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [October 18, 2019, 6:25pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/18 "2019-10-18T18:25:54Z")

</div>

Side note and way to late now but why do we not have something like `STAN_THREADS=true` like we do for OpenCL? I find that much simpler to explain and use.

---

<div class="post-metadata">

**Author:** ![seantalts](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/seantalts/32/32_2.png) [@seantalts](https://discourse.mc-stan.org/u/seantalts)\
**Post date:** [October 18, 2019, 6:32pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/19 "2019-10-18T18:32:23Z")

</div>

We should add that :)

---

<div class="post-metadata">

**Author:** ![stevebronder](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stevebronder/32/17242_2.png) [@stevebronder](https://discourse.mc-stan.org/u/stevebronder)\
**Post date:** [October 18, 2019, 6:33pm UTC](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121/20 "2019-10-18T18:33:43Z")

</div>

Should probs talk about this on a seperate thread but I’d really like to make the OpenCL and threading stuff command line options

[Next page](https://discourse.mc-stan.org/t/stan-math-release-notes-draft/11121.md?page=2)