Saving progress and sampling for more iterations

I’ve seen a small number of posts about this from years past, and am wondering if there has been any recent development in rstan towards being able to 1) save the progress of a model that is still warming up or sampling, and 2) be able to have a model sample for additional iterations after finishing when there are rhat or ess warnings without starting over entirely. I’m a windows 10 and R user and mostly relying on BRMS. As with the few other posts I was able to find about this, I often have no control on when my jobs get canceled and have models that take days or weeks to run. Any advice is much appreciated!

Automated solutions to this are in the works. What you can do in the interim is have brms give you the Stan code and data json files for the model you specify, then use cmdstanr to start the chains while telling it to write the chains’ output to csv (it actually always does this, but by giving it an explicit location, they get saved in a non-ephemeral location). Then if the sampling ends unexpectedly for any reason, you can use/extend the code here to resume (though you may need to work out how to parse an incomplete csv; I’m not sure if cmdstanr can do that automatically).

1 Like