MPI framework for parallelized warmups

Changing adaptation is more subtle than just halving the length of the chains due to the initial (for finding the typical set without any adaptation) and terminal (for finalizing the step size) windows and the multiplicatively expanding adaptation windows between (for aggregating variance information to inform updates to the inverse metric components and updating the step size after the previous update).

Sure, just be aware that the approach that evolves with your prototype might not be chosen so there’s the risk of redundant work. Developing that prototype will definitely help inform the design you want, but like any exploratory development it brings with it the risk of not ultimately being used.

The exact service routes will have to change a bit. Firstly they will have to be able to run multiple chains, then break up warmup into chunks to allow intrachain communication.

Which to prioritize is totally up to the person developing the design.

Personally I think that we want to prioritize chain parallelization first because it is beneficial in of itself.

  1. Service routes that run multiple chains and create multiple streams of output.
  2. Parallelizing those routes, allowing all of the interfaces consistent parallelization without having to implement it themselves.
  3. Progressive warmup to allow for intra chain communication.
  4. Adaptive warmup through that intra chain communication.