Following the discussion at https://discourse.mc-stan.org/t/promoting-posteriordb-into-an-official-stan-project I am calling a vote on the inclusion of
posteriordb as a core Stan project. For those who didn’t follow the original discussion, here is what it would mean and the main disagreements that remained. A good summary of the main arguments and points of disagreement is IMHO this post by @mans_magnusson.
To very briefly summarise the summary:
posteriordb is designed to test inference algorithms across a wide range of models and data sets. It stores models and validated results to compare against and provides API to access them. Everybody agreed that having such a database is useful for Stan.
The only major disagreement from the discussion was IMHO about scope of the project, specifically about the potential for scope creep/feature bloat. The points in this direction were made by @betanalpha. If I understand Mike correctly, he would like a relatively tight specification of the project and its goals before promoting it. If I understand @mans_magnusson and others correctly, they consider such tight specification premature and not necessarily useful/desirable and are OK with a higher-level description allowing for more agility in future development, e.g. “Testing implementations of inference algorithms with asymptotically decreasing bias and variance (such as MCMC)” and “the goal of
posteriordb is the best possible Bayesian inference” (the goals are currently summarised at https://github.com/MansMeg/posteriordb/blob/master/doc/use_cases.md).
There was also a concern about package architecture especially R-first + interfaces for other languages (current state) vs. full plugin architecture (proposed) and about code sharing/dependencies. Those were not completely resolved either way, but I personally believe such development decisions can safely be considered a day-to-day devel business and not be decided together with acceptance (your opinion may vary).
As a consequence of accepting the project, its developers will become Stan developers (and thus e.g. eligible to vote). The main
posteriordb developer who is not already Stan developer is @mans_magnusson. There are other non-trivial contributions (jarnefeltoliver, eerolinna, who AFAIK are not users on Discourse), but since it is unclear whether they want to become part of the Stan developer community and their contribution is not obviously fundamental, they would go through the usual process (which is currently not completely defined) of approving new developers, should they want to join the ranks.
An additional goal of the vote is to record consensus on acceptance (if it is there).
The discussion leaves us with three options to vote on:
- Promote posteriordb into an official Stan project in its current state and goals and plans as outlined in the discussion thread.
- Promote posteriordb into an official Stan project conditionally, only if its scope, goals, implementation or plans are made more specific and further narrowed only to those immediately relevant to core Stan mission. A further discussion would follow on how exactly should this ‘narrowing’ look like and whether the posteriordb team would like the project to join Stan under such conditions.
- Do not promote posteriordb into an official Stan project
The voting period is two weeks, but with regards to holidays in some parts of the world, I would push it back a few days, so that voting is until January 9th. If any option gets more than 50% votes (not including pepole who abstain from voting) it is selected, otherwise there will be a second round among the top two options. Tagging the whole @Stan_Development_Team to consider voting, if they feel sufficiently informed about the issue (this vote is not restricted to a specific module).
The voting will be done with Discourse poll below (the options are shortened compared to the full version above, but the full version is important).
- Promote posteriordb as is
- Promote posteriordb conditional on narrower scope/further clarification
- Do not promote posteriordb
- Abstain from voting
The “Abstain” option is the same as not voting at all, but it lets me understand how many people missed this post (and thus I should nudge them to vote) versus how many just don’t want to express opinion on the topic. You obviously don’t have to participate in any way.
Thanks everybody for their participation and patience so far.