Checkpointing with CmdStanPy

nerpa · September 24, 2020, 4:26pm

Hello all,

Following the thread on checkpointing with Stan, is there any tutorial that was created on this? My wall time limit is 24 hrs and the full model takes a few days to sample (using reduce_sum with 90 cores).

Thanks!

ahartikainen · September 24, 2020, 4:58pm

Hi, I’m not sure if there are any official tutorial

Here is one example with PyStan + ArviZ

github.com

ahartikainen/fit_check_fit_loop/blob/master/MoreSamples_round1/increase_draws_in_steps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example how to iteratively get more draws (on PyStan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pystan # current develop branch (2019-06-28)\n",
    "import arviz as az # current master branch (2019-06-28)\n",
    "\n",
    "from tqdm import tqdm_notebook as tqdm\n",
    "import numpy as np\n",

This file has been truncated. show original

And also here

Both of these assume that warmup can be done in 1 step (1 run).

nerpa · September 24, 2020, 5:21pm

Thanks! I wish there could me such a nice example with CmdStanPy as well!
What would be the inv_metric equivalent from the cmdstan csv output file?
Is last_position simply the final value of the cmdstan output file for each parameter?

Thanks!

Topic		Replies	Views
Current state of checkpointing in Stan Developers features	27	3257	November 18, 2020
Sampling chains from the middle in cmdstanpy General	10	555	October 8, 2020
Checkpointing CmdStan sampling Interfaces	24	2335	June 30, 2020
Which Stan interface supports checkpointing? General cmdstan , rstan , stan , cmdstanr	6	990	May 5, 2021
CmdStanPy - ready for beta testing! Developers pystan	23	2329	August 6, 2019

Checkpointing with CmdStanPy

Related topics