Checkpointing with CmdStanPy

nerpa · September 24, 2020, 4:26pm

Hello all,

Following the thread on checkpointing with Stan, is there any tutorial that was created on this? My wall time limit is 24 hrs and the full model takes a few days to sample (using reduce_sum with 90 cores).

Thanks!

ahartikainen · September 24, 2020, 4:58pm

Hi, I’m not sure if there are any official tutorial

Here is one example with PyStan + ArviZ

github.com

ahartikainen/fit_check_fit_loop/blob/master/MoreSamples_round1/increase_draws_in_steps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example how to iteratively get more draws (on PyStan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pystan # current develop branch (2019-06-28)\n",
    "import arviz as az # current master branch (2019-06-28)\n",
    "\n",
    "from tqdm import tqdm_notebook as tqdm\n",
    "import numpy as np\n",

This file has been truncated. show original

And also here

Both of these assume that warmup can be done in 1 step (1 run).

nerpa · September 24, 2020, 5:21pm

Thanks! I wish there could me such a nice example with CmdStanPy as well!
What would be the inv_metric equivalent from the cmdstan csv output file?
Is last_position simply the final value of the cmdstan output file for each parameter?

Thanks!

Topic		Replies	Views
Sampling chains from the middle in cmdstanpy General	10	510	October 8, 2020
Current state of checkpointing in Stan Developers features	27	3076	November 18, 2020
Checkpointing CmdStan sampling Interfaces	24	2125	June 30, 2020
Which Stan interface supports checkpointing? General cmdstan , rstan , stan , cmdstanr	6	884	May 5, 2021
CmdStanPy - ready for beta testing! Developers pystan	23	2166	August 6, 2019

Checkpointing with CmdStanPy

Related topics