Monthly Math development meeting: 06/25/2020, 10 am EDT

@wds15: ODEs. Move sensitivity to error checks. Is there a way to check it out? @bbbales2: don’t have enough time to shove it to the future. Use current option and go with that for now. @bbbales2: is the time step dependent on the tolerances of the sensitivities?

@wds15: asking @rok_cesnovar about changing init threadpool for CmdStan. init threadpool method was deprecated in the next version. @rok_cesnovar: do we want Stan Math to have a convenience function to init? Will look into an easier way of doing this.

@wds15: looking at ODE stuff, looking at error checking function. There’s a global function. Concern: globals bad for threading. @syclik: I will take action and document this (stan::error_index::value).

@stevebronder: Next release? @rok_cesnovar: 18 July, but that’s Saturday, so 20 July. 13 July (+/- day or two) will be code freeze.

@rok_cesnovar: asking about 3 releases a year instead of 4. All: just discussing the different things getting into this release.

We’re discussing the unnormalized distributions: Request for final feedback: User controlled unnormalized (propto) distribution syntax

@rok_cesnovar: new Windows RTools pass and old RTools pass. It seg faults due to large size.

@wds15: there’s more MPI code coming. How do we manage it? Looks like @yizhang wants to lead. Is ok taking position of removing MPI. @rok_cesnovar: the first problem is that not enough people know about MPI.

Just to clarify: I am not proposing to remove MPI… I am just worried about its future maintenance. I will likely draft a rough design doc of the current MPI system to make it easy for anyone to take it on.

2 Likes

Meeting adjourned! We’ll meet next month at the same time on the 3rd Thursday.

Following up on the var branch stuff, I made a post in the pull.

The vague plan was to merge it in after 2.24 if it looked like we could get at least one of the follow up pulls in too (sparse matrices or static matrices or whatever) so we’re not leaving a pull with 50 changed files open (cause those are kinda hellish to handle the merge conflicts on).

I can only confirm that I’m working with @bbbales2 & @avehtari on MPI implementation of cross-chain warmup algorithm. I’m agnostic of whether to remove MPI from Stan.

Not sure if this is the right place to ask or whether we should spin off a separate thread, but is there anything MPI gives us that tbb does not? It sounds like we want mpi for communication around a cluster of computers while tbb is for communication across threads for one computer. Or is that false?

In general they are for orthogonal parallel paradigm: distributed vs SMP(shared memory). Specifically,

yes, to do distributed computing.

We want mpi for communication across cluster nodes, that’s for sure. As to parallelization within a single node, MPI can do it but it depends on actual circumstance we may choose it or TBB. TBB is for single node shared memory multithreading, and not suitable to communication cross nodes.

1 Like

So you would be OK to ditch MPI from Stan-math? How would cross-chain things then happen?

Good question. If we are ok to say that the CPU cores from a single machine will always be sufficient to drive a single chain, then one could actually ditch MPI from Stan-math and only use MPI as a wrapper around all of this (like the cross-chain thing).

Anyway, I don’t want to get into yet another flame war, really. As I said, I will write up some high-level bits and pieces about MPI in Stan-math. With that document anyone else can hopefully take this stuff on to wherever it should go to.

Yes.

Stays in Torsten.

I’m fine with waiting on this, afterwards it may be good to have a stan math meeting to talk about all this. I’m pretty indifferent one way or the other

I think this convo is just about remove mpi from stan math. I think cross communication is exactly what we want when working across multiple chains so mpi should stay in the Stan repo!

It’s not my intention to remove it. It’s my intention that the project can keep maintaining it in way which does not necessarily involve me in case MPI is being pushed forward to handle more thing. At least that is my goal and writing a design doc on my end should get us there.

However, it is still a good question to ask: If MPI stays within the Stan repos - in which one? Does it really have to be in Stan-math? I don’t know.

@yizhang The cross-chain warmup is likely very useful to everyone. The current prototype is based on a very different MPI backend, etc. Are there already plans how this is being moved into the stan repos? A refactored MPI in Stan-math, a thread implementation - what are the plans in this regard? Or should we revisit these questions once things are mature enough? And if so, how close are we to that level of matureness from your perspective?

This discussion needs to continue in a new topic to avoid hijacking current thread, also in order to involve others.

2 Likes

Go ahead.