Monthly Math development meeting: 9/19/2019, 10am

Hey all,

The next Math meeting is Thursday, Aug 19, 10 am Eastern time. This will last no more than one hour; if we need more time, we will schedule additional time.

This meeting will be open to all those that currently contribute to the Math library or that would like to contribute to the Math library. If you’d like an invite to the Google Calendar event, please DM me with your Google handle. If you’d like to just have a copy of the event on your calendar, please use this link:


  • this is a chance to collaborate and coordinate efforts. We’re expecting everyone to be kind and courteous. And patient. We have a team with diverse backgrounds.
  • impactful decisions will not be made at these meetings. That will continue to be done online on these forums. The meetings are a good time to discuss and get on the same page, but the decision making will be done online where we can include the opinions of others that can’t make the meeting.
  • important discussions that happen at the meeting will be communicated back online on these forums. If they don’t make it back here, it wasn’t important. No one will be penalized for not being present at a meeting.

If there’s anything you know you want to talk about, please post here. This is a good time to request help from others if there’s expertise that you need.

Agenda items (please add any additional as comments and we can update over the week):

  • Apache 2 licensing. Summary of where we are, what decisions we can make, when we can make them. How and when we should go through with Intel TBB.

  • Next release. Code freeze, Oct 11. What version we go with.

  • Policy updates: we’re waiting on exact wording, but we can talk about what we’d like our development process to be and we can work toward creating a policy that makes sense.

  • Asks: if anyone has asks.

I will try to be there and I can make suggestions as to how to include the TBB. My thoughts (to be discussed)

  • bump version to the current 2019 update 8
  • enable tbbmalloc if the user defines STAN_THREADS
  • refrain from a script since we move to conan soon(?)

Related to that we should think about how the task_arena’s are managed. So these control the size of the thread pool. Right now we basically control the number of threads in Stan-math… but I think that sessions management should be moved outside of Stan-math. Basically this means the math library will assume to run inside a task_arena such that resource restrictions like number of concurrent tasks are handled outside of math in the whatever wrapping program. We don’t need to settle this now, but it would be good to think about it.


@syclik would it help if I wrote down the “meeting minutes”? You are usually being asked to chime in on questions from other participating developers, so that might help you have more time to answer questions by not needing to write down stuff?

Absolutely! An alternative would be to rotate note takers. But, yes, for the next meeting, that would be appreciated.

There is an update on this. Just linking here so we have it handy for the meeting.

1 Like

If we have time, I’d like to start discussing performance testing. We could use a brainstorm session to come up with requirements.

Devs present: @syclik, @wds15, @Stevo15025 and @rok_cesnovar .

  1. TBB

TLDR: Green lights for TBB all around. It will be merged for 2.21 if we can also enhance map_rect with TBB. Resource managment will be handled on the Stan level with a possibility for Stan Math helper functions if they can be done reasonably.

1a) Licensing:

We did a summary of what has been a discussion in various places which is:

Any project is fine if they don’t distribute a Stan model executable and are GPLv2

Daniel said that he feels we should include a huge warning that we are including an Apache 2 licensed dependency in the release we do it. We all agreed that it should definitely be visibly noted.

1b) Merging TBB:

Sebastian said that together with the TBB merge he will enable the use of tbbmalloc when STAN_THREADS are on. If STAN_THREADS is not on, there are some worries of a performance degradation on AMD CPUs (5% was the first figure Steve got, but he said it was not a thorough test). Rok will prepare an update script to update script to the latest version (and test it on the PR).

1c) Timeline

We had a discussion on when do we merge TBB, 2.21 or wait for 2.22. Daniel’s point was to make 2.21 the last “Apache 2 dependency free” release. There was some more discussion there but the final conclusion was if Sebastian can improve map_rect with TBB to enable scheduling (and something else I forgot to write down) we then release it in 2.21. If not and TBB would only be added but not really used outside tbbmalloc we wait for 2.22. Sebastian noted that if we released it for 2.21 that would give us 3 months of testing it in CmdStan and we would see if its really as advertised before we move it to the other interfaces.

1d) Resource managment

Sebastian wanted to discuss how to do resource management with TBB and limiting it with STAN_NUM_THREADS. He mentioned that he felt that it should be done at Stan level, not Stan Math. We were all in agreement that would make the most sense. We then discussed how things would work for Stan Math standalone. Sebastian mentioned that if nothing was explicitly done, the default resource management would be in effect with the Stan Math user having the option of modifying it via TBB. Rok asked if there is an option of adding a helper function in Stan Math to make things easy. That opened some discussion for which the final conclusion was that we should aim for a helper function for resource management in Stan Math if its reasonable to implement it, otherwise at least have some instruction ready on how a user can do it with TBB.

Steve is going to send the TBB book to Rok.

2.) Feature freeze

Everyone is OK with the feature freeze. There was some discussion on how we will handle PRs between the 11th and 18th. Will we restrict merging or tag develop on the 11th? Daniel said that we can go whichever route we want.

  1. PR policy

Daniel does not like having PR czars that Steve mentioned in the PR reviewing issue. Steve clarified that the czars would be more of a list who to tag if you have no idea who to ask for review. Daniel was OK with that. We then talked about policy at length. Daniel does not like the idea of undocumented functions. We should discuss that some more on the discourse topic. Then put it in the contributing on github after we have a “vote”.

  1. Performance testing

We had a debate on the required amount of performance testing for PRs, that we should aim for always have faster functions across the board, but acknowledged that there are cases of tradeoffs between speed and readability. Daniel mentioned that we should have more discussion regarding how to do performance testing on the function level. Sebastian noted that the focus should be on the performance impact on the Stan model level rather than function level. No real conclusions were made in this debate, just that we should discuss this more on Discourse.

  1. Other

Sebastian mentioned he is trying out AMICI ( for ODE solving, that it looks fast (I think a 2-fold speedup was mentioned).

If I missed anything or got something wrong let me know.

EDIT: I seem to not be able to spell management :)