We're starting a trial period for the development voting procedure!

@Stan_Development_Team and Stan community,

On behalf of the Stan Governing Body (@SGB) I am happy to announce that we’re now starting a trial period for the proposed voting procedure on technical issues. The text of the proposal can be found in the linked topic and also copied at the end of this announcement.

Thank you to everyone for the very helpful suggestions. We made many edits to the initial proposal in response to your feedback (see linked thread above) and during the trial period we encourage you to continue to share your feedback. In fact it is essential that you do because we don’t want this to feel like something imposed on you by the governing body but rather something that we all create together for the benefit of the project. If you have any concerns that your feedback hasn’t been incorporated please let us know because it is almost certainly an oversight (you can contact us privately via board@mc-stan.org or publicly here on the forums by tagging @SGB). We attempted to incorporate or respond to every suggestion we received but it’s possible we missed something.

Ok, now the details:

It was not obvious to us (the SGB members) how long the trial period should last because it is hard to anticipate how often this new procedure will come into play. We decided that we will aim for 3 votes or 4 months, whichever is longer. Our reasons are:

  • we want at least 3 votes to be able to assess the merits of the system
  • the ebbs and flows of software development make it possible that we could have a stretch of months where there are no issues major enough to require a vote
  • if there happen to be 3 votes in a very short amount of time we still want to give people more time to think about what they like and don’t like about the process
  • we don’t want this to drag on forever, so we will cap it at 4 months (which will be early 2021)

But this is not set in stone, so if you have suggestions for a better timeframe please let us know and we can assess whether it’s worth changing the parameters of the trial period.
The current timeframe also means that a new SGB will be elected before the trial period is over and that they will then take over management of the trial.

Thanks again for everyone’s help drafting this proposal and we sincerely hope that this will be beneficial to Stan development going forward.


Proposed voting procedure (click to view)

SGB proposal for voting on technical issues

copied from Revised SGB proposal for voting on technical issues

Background

As it stands right now, Stan does not have a formal process for managing disagreement on technical issues. Although the project has been successful to date, there is a benefit to having an agreed-upon procedure in place to manage serious disagreement on technical issues. To this end, the SGB is aiming for a lightweight solution that is not burdensome for developers, but that will provide a mechanism for reaching a decision when informal consensus appears unlikely. Visibility into how technical decisions are made also helps onboard new developers to the project.

Below is a proposal for a voting system for making technical decisions in case of disagreement among developers.

When should you call a vote?

A vote is only necessary if there is still disagreement on a major issue after ample discussion. For the most important decisions, a vote should be called even if there is no disagreement in order to create a record of the consensus.

We realize there is ambiguity on what constitutes a “major” issue or “ample” discussion. Some judgement is required on the part of the initial developer. A working definition is that an issue is “major” if it may create a substantial maintenance burden on other developers or introduces any user-facing changes that may be controversial (of course there is ambiguity about these definitions too). “Ample” discussion means that the issue has been raised, and other developers have had an opportunity to read, process, and chime in. If other developers think the vote was called too soon then they can vote “No”. A proposal voted down can be brought up for a vote again after subsequent discussion.

If you’re not sure if a vote is required, err on the side of collaborative decision making by calling a vote. However, keep in mind that voting is intended as a fallback and you should first make an attempt to understand and address the technical concerns through polite discussion.

Who can vote?

  • Votes are open to developers contributing to the relevant module (see bullet point below) but any developer outside the module can force it to a developer-wide vote by objecting. Developers calling for a vote also have the ability to nominate multiple relevant repos initially.

  • A module is a repository on stan-dev. (This is a starting point. Depending on how this goes and how often modules need to be grouped together this can be revisited.)

  • The exception to this is for major changes to the Stan language, in which case:

    • The proposed change should be posted on Discourse to allow the entire community to comment
    • The voting will be open to all modules.
    • Note: this is for major user-facing language changes and does not pertain to implementation issues, which are the purview of the language module developers. An example of a major user-facing change is a totally new syntax for declaring arrays (New array declaration syntax). An example of a minor user-facing change/addition is vectorizing binary functions (https://github.com/stan-dev/stanc3/issues/643).
  • If you are eligible to vote it is up to you whether or not to vote, but please only vote if you think you are sufficiently informed about the relevant issues.

How does voting work?

  • Platform for voting: This is TBD, but for the initial trial period we will try out the “poll” feature on Discourse (https://meta.discourse.org/t/how-to-create-polls/77548). Other suggestions are welcome.

  • The following procedure will be used:

    • Someone proposes a vote and contacts SGB
    • The SGB appoints someone to run the vote or recommend to the SGB that the vote not happen
    • That person sets up the post with the poll, notifies voters, and reports the results after the vote (the fact that a vote has been requested and the final outcome of the vote should be made public).
  • A poll is open for 2 weeks in order to give everyone enough time. If a lazy vote fails and a majority approval is required, please create a new poll and note that it is requesting majority approval.

  • Three vote options are available:

    • +1: Agree
    • 0: Abstain (the default)
    • -1: Disagree (you must list your reasons for disagreement in a comment)

    (We may need to adjust this if a particular topic in question can’t be expressed as a yes/no vote and requires more options.)

  • Proposal phrasing: Please include a statement that makes it clear what it means to “agree” or “disagree.”

  • Lazy vote: A binding agreement is reached when there is a minimum of one +1 and no -1s. In other words, if anyone is in favor and no one is against then the vote passes.

  • Majority approval: If a vote does not pass after a lazy vote, then the issue will require majority approval, where a binding agreement is reached when there are at least two +1s and more +1s than -1s. However, majority approval should be considered a last resort. The aim should be to reach rough consensus through discussion. To facilitate reaching rough consensus, voters are allowed to change their vote during the voting window if they change their minds.

  • Every voter’s vote has equal weight and each person can only cast one vote on a given call. (The developer bringing up a topic for a vote can also cast their vote.)

Arbitration

The voting procedure is intended to help resolve developer disagreement but it is possible for disagreement to happen about the process itself. In this case there is the option for arbitration, which works as follows.

  • If at least 3 developers consider any part of the process leading to a particular vote problematic, they may ask the SGB to arbitrate. An arbitration request stops the voting period if it already started.

  • The SGB may designate a community member to serve as an arbiter either for a specific case or to generally handle all arbitration requests for a module. The arbiter will act on behalf of the SGB and notify the SGB of the resolution of any arbitration.

  • The arbiter/SGB will consult all parties involved and then make a final decision on how the vote should proceed. That is, the arbiter/SGB will not decide the technical issue itself, but rather resolve issues related to the process, e.g., when a vote is called, the order of voting on multiple proposals, the final wording of the proposal, etc.


15 Likes

One question the recent vote (Voting on inclusion of posteriordb) has brought into consideration is who should call a vote? My understanding was always that the parties involved in any discussion would be the ones responsible for elevating an issue to a vote where as here @martinmodrak has called the vote external to the discussants, and without any agreement on the terms of the vote from the discussants. In hindsight this wasn’t discussed at all in the voting procedure, but I think would be important in future iterations.

1 Like

Thanks for pointing this out. I didn’t realize we forgot to discuss this but you’re right. I agree we should for future iterations.

1 Like