TWG Definition/Roles. Call for feedback/ideas/refinement

[breckbaldwin]

First off thanks for all the thought that has gone into this thread. This is exactly what I was hoping for. I’ll start at the top.

The CTO position was an attempt by me and someone else, who may out themselves if they choose, to get Bob back into the decision making process. It has failed and I remove it.

@anon75146577

The biggest responsibility is keeping PRs (pull requests which are software changes waiting for review) moving in a timely manner. This means approving PRs, requesting changes to the PRs, marking working in progress on PRs or closing ideally within one week of submission. Delegation is highly encouraged.

What happens when a lead fails to do this?

They need to be replaced by the SGB if PRs don’t flow. We have multiple instances of developers getting frustrated by the tech leads and this has to stop. We are very dependent on external to Columbia contributions, thanks folks!, and throwing up barriers is not good. Structures that make sense with 4 folks in an office banging out Stan 1.0 changes when we have 20+ contributors.

@Bob_Carpenter

Why is the TWG job so hard?

I think the TWG director has proven to be a difficult job because it’s the wrong way to run an open-source project. I’m largely responsible for the current direction, with help from @seantalts. We both thought having a management structure more like the good companies we worked at would be more efficient in decision making.

We tried something and it is not working out in certain respects. I’d like to see a more community building role from the TWG director which is borrowed from James Vasile’s advice (OpenTech Strategies). I don’t want to toss the single point of responsibility for the technical side yet. I think it almost worked this time.

Happy to discuss further.

@Bob_Carpenter

While it would be nice to have efficient top-down decision making, I didn’t at all anticipate the hard feelings this would create in the community.

Having been through a few controversies in the past year I’d say the trigger points for controversy are:

1 Lack of transparency
2. Lack of awareness that decisions are being made
3. Lack of being heard.

Addressing each in turn I’d say that 1) transparency has increased if you know where to look which is what 2) is about. I think we need a weekly summation of what is going on, what needs action in Stan and it should happen at the weekly meeting Thursday. 3) is what really drives the frustration, we do need to hear from everyone and make sure they know that.

I did decide to try something different with the Apache licensing conversation as Bob notes:

@Bob_Carpenter

A nice example of how this works is the way @breckbaldwin managed the Apache 2 licensing discussion (not related to Apache process). That felt open and inclusive, yet it came to a conclusion relatively quickly.

Thanks for identifying something that is working. A very hard part of this job is all the complaining about how ideas don’t work. You are a tough group to work with.

Back to the process, all I did was restate what @seantalts was driving for, I just crisped it up. It was not quick in my estimate, but it is the model I’d like to pursue. I came up with a tripwire version that if anyone objected to adoption we would go to majority vote (Apache 2 library inclusion into Stan Math--general policy case. Trip wire consensus decision). Rough consensus sounds good too.

The decision making process I’d like to see is;

1. Announce upcoming decision at general Stan meeting.
2. Discuss on discourse for a week.
3. Talk about at next weeks Stan meeting
4. Discuss for a few more days, kick to vote/tripwire/rough consensus/sgb depending on the issue
5. Done in two weeks ideally

I am trying this with the TWG issue we are discussing now. I’d consider last week’s meeting to be 1), doing 2) now and 3) tomorrow (Stan General Meeting Thursday September 26, 2019 11am EDT). The final decision is actually the SGB’s to make, see Your new governing body.

@Bob_Carpenter

In or out?

I had originally cordoned off the downstream interfaces like rstanarm and brms because I didn’t feel like I understood them well enough and because they seemed to require much less coordination because they are endpoints rather than infrastructure, so changes to them don’t have knock-on effects. I also felt I had no hope of influencing rstan or pystan given how many battles I’d lost in the past on API design.

But that was just my failure. Going forward, I would very much like to see all of the projects branded as Stan brought under the same process and governed in the same way.

This is very much my personal opinion but Stan’s responsibilities need to simplify. The downstream packages like RStanArm, BRMS and my favorite ShinyStan are bigger than Stan org. They are welcome in our repo however.

@Bob_Carpenter

I don’t think there’s much to do for algorithms as we are basically just telling people to go away until they’ve proven their algorithm works; here’s a quote from the Wiki page.

With so many algorithms introduced each week the Stan development team unfortunately does not have the resources to assist with this testing, and consequently the evidence burden is the responsibility of anyone who might want their method to be considered.

I think this is a big decision that we need to think hard about. I’ll remind all that both @andrewgelman and @avehtari are very focused on inference research while at the same time being huge Stan supporters with research groups that build Stan. We have other research groups sniffing around looking for a happy place to do inference research.

I believe that at my first Stan meeting, essentially my interview, that I actively wondered why there is not a plugin architecture for inference research, gradient based and otherwise? Not doing so given the psychological energy telling people to go away seems odd. I’ll leave this for future discussion.

@anon75146577

Why was the TWG director singled out for extra compensation?

My understanding from the meeting where this was decided was that the TWG lead was expected to have to take responsibility for some unpopular decisions and was basically being paid to take the flak. Think danger pay, but where the danger is snarky emails from men in their 30s who shouldn’t send snarky emails. I don’t think this is what played out and I hope the SGB is reconsidering things as they refactor the post.

Paying the TWG director was my idea. On any version it is a crap job. I’d like to remind all that economics matter.

So why pay people?

• Rich people can volunteer, poor people cannot. If you want diversity, pay people.
• These jobs are “work” as in not done for fun.
• The jobs are ludicrously underpaid ??$10k for a youtube manager??, more like a “tip”, for expected value. Eventually we start paying people for the value their jobs bring.
• We have the money, this is not a struggle for Stan and we can experiment with what works.

This is a great discussion and what we can use. I hope I have been respectful of opinions I agree and disagree with.

Breck