Good point. This seems like a decent way to deal with that for now.
We all want to get MPI in as soon as possible. We’re very excited to be able to scale out in a way we never could before, and doing that with generality is very exciting. The design review is exactly about the internal design and architecture - making it first and foremost easy to understand for maintenance. This involves creating layers of abstraction that hide details away so someone who doesn’t know the codebase can come in and understand a minimal subset of the layer they need to work on. We can have a conversation about the best way to do that, but that is the objective here.
For 5 - Gotcha. The MPI stuff I looked at didn’t have this same client-server style architecture. I think I understand why we want that - we need the root node to guide HMC exploration and the children to do whatever work the root decides needs to be done, so the root could be branching based on parameters or what-have-you. Cool.