Multicore Speedups are different between models

[betanalpha]

@anon75146577

Because the chains find their own diagonal mass matrix during the warmup phase, they also won’t really be running the same Markov chain (the scaling will be different), so it’s possible that one chain will be scaled worse than the others and will therefore take longer for the modified NUTS criterion to tell it to stop exploring along the trajectory.

This can happen, but if the sampler is well-behave then the differences from chain to chain should be small.

@aaronjg

There is no coordination between chains. If you are using rstan (what I am most familiar with), you can run each chain independently, and then combine the results at the end - if you pay for compute resources this is going to be the most efficient way to do it.

You can do this only if each of the individual chains are well-behaved.

When you run MCMC you have to be very careful that you’re getting an accurate answer, which is hard because there’s no way to prove that a given sampler will be accurate for your specific model. Instead all we have are conditions that we know should not happen for well-behaved chains (i.e. we have necessary conditions but not sufficient ones).

One of the common ways that these pathologies manifest is chains behaving differently depending on where they are in parameter space. This is why we run many chains and compare them with R-hat – if behavior of even one chain deviates from the behavior of the others then we should doubt the validity of all of the chains.

So if you run multiple chains and you see drastically different speeds then it’s likely your sampler can’t handle your model. To verify this check out the R-hats as well as all of the other diagnostics (especially divergences). If the diagnostics look fine and the speed differences aren’t huge (between 1 and 2) then the variation is likely due to small differences in adaptation or core performance, in which case the variations can be ignored.