Chain Parallelization in Stan

[jjramsey]

@ptheguy

I read here that within-chain parallelization is not supported in Stan due to the mentioned reasons.

I think this is somewhat outdated. IIRC, map_rect handles within-chain parallelism.

@ptheguy

Does this mean Stan supports parallelization between chains? If yes, is this done automatically

It’s not so much parallelism between chains, since each chain is independent of the others. There’s no communication between the chains.

@ptheguy

I’m running on a cluster, which is based on Slurm. I’m essentially wondering if I can be as relaxed as simply issuing the Slurm flag --cpus-per-tasks=10 and have Stan take care of the parallelizng the chains, or I need to do something more involved.

It’s a little more involved, but not by that much. If you aren’t bothering with map_rect, then you can tell your Stan implementation to create as many chains as you have cores available. That may involve, say, passing $SLURM_CPUS_ON_NODE to Stan somehow, which will depend on which implementation you use, e.g. RStan, PyStan, CmdStan, etc.