Checkpointing with CmdStanPy

nerpa · September 24, 2020, 4:26pm

Hello all,

Following the thread on checkpointing with Stan, is there any tutorial that was created on this? My wall time limit is 24 hrs and the full model takes a few days to sample (using reduce_sum with 90 cores).

Thanks!

ahartikainen · September 24, 2020, 4:58pm

Hi, I’m not sure if there are any official tutorial

Here is one example with PyStan + ArviZ

github.com

ahartikainen/fit_check_fit_loop/blob/master/MoreSamples_round1/increase_draws_in_steps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example how to iteratively get more draws (on PyStan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pystan # current develop branch (2019-06-28)\n",
    "import arviz as az # current master branch (2019-06-28)\n",
    "\n",
    "from tqdm import tqdm_notebook as tqdm\n",
    "import numpy as np\n",

This file has been truncated. show original

And also here

Both of these assume that warmup can be done in 1 step (1 run).

nerpa · September 24, 2020, 5:21pm

Thanks! I wish there could me such a nice example with CmdStanPy as well!
What would be the inv_metric equivalent from the cmdstan csv output file?
Is last_position simply the final value of the cmdstan output file for each parameter?

Thanks!

Topic		Replies	Views
Sampling chains from the middle in cmdstanpy General	10	419	October 8, 2020
CmdStanPy - ready for beta testing! Developers pystan	23	2066	August 6, 2019
Transitioning from pystan 2.x to 3.x PyStan	3	299	November 30, 2023
CmdStan slower than PyStan? Interfaces cmdstan , pystan	26	2359	April 17, 2019
Warmup iterations in PyStan 3.7 PyStan	5	463	May 11, 2023

Checkpointing with CmdStanPy

Related Topics