Resume sampling after interuption

It’s starting off from something that finished sampling.

It looks reasonable, but a couple problems:

  1. If warmup hadn’t finished, then you wouldn’t have a metric.

  2. I don’t know if there’s any way to cleanly stop a cmdstanpy job if things get interrupted partway through (I don’t know how spot instances work).

  3. I’m scared partial output might break this process.

There’s a checkpointing thread over here that has some info: Current state of checkpointing in Stan - #13 by bbbales2

There is something in cmdstan called the diagnostic file (I think recently [last few days] it has been renamed the latent dynamics file, but I don’t know if that’s made it to cmdstanpy yet) that could be used to do this.

@mitzimorris is there a way to provide a stepsize per-chain in cmdstanpy?

1 Like