Subclassing Cmdstanpy

[Funko_Unko]

@mitzimorris

what do you mean by “restart sampling”?

• continue adaptation? in which case, initializing with stepsize and mass matrix should work.
• get more samples? initializing with stepsize and mass matrix, no_warmup, will produce more samples. but there’s a problem in that we don’t save the state of the RNG - the seed + chain_id is the initial offset, but then during a run, we record how many times it’s called. to properly continue, one would need to know both the seed and how far to advance the RNG.

I mean restarting with the current metric, step size and points in parameter space. There were three issues:

• Couldn’t initialize stepsizes with an np.ndarray (this appears to be fixed)
• Couldn’t initialize inits with a list of dicts, added an issue here Can't initialize chains with different inits via list of dicts. · Issue #362 · stan-dev/cmdstanpy · GitHub
• Couldn’t initialize metrics with a list of np.ndarrays or with a single one. Didn’t add an issue.

The last two points aren’t really advertised functionalities of the sample method, but this sure would be convenient (at least for me).

Anyhow, what I did in subclassing the CmdStanModel was just hacking together the appropriate json serialization.

@ahartikainen

In Stan code?

Hm, I do not believe it is possible to infer the size of an array in Stan, or is it? Meaning for example having a data block

int no_elements;
vector[no_elements] v;

and only specifying v? Because I’m lazy, I didn’t want to always type (in python) data=dict(v=v, no_elements=len(v)) but just data=dict(v=v) and let no_elements be inferred from the shape of v. It’s of course not only about the shape of arrays, but still probably not a mainstream functionality.

@mitzimorris

we recommend running multiple chains in order to be able to test convergence. because if you don’t have convergence, whatever your sample is, you have no evidence that you have a set of draws from the posterior, consequently, no reason to believe that the samples are valid.

therefore, you need to wait until all chains have finished sampling. granted, this is problematic if you’re not able to run all chains in parallel or otherwise distribute them across a cluster.

Yes, the use case is not to just discard the aberrant chains and act as if everything was fine. Instead I have the following situation:

I have a reference solution/sample/method, and want to compare it to other methods. For some of the methods, there arise issues during sampling, leading to chains gettings stuck and taking super long. Eventually, they may find the other chains, or they may not. In either case, I wan’t to be able to terminate the chain, and work with the data that I have, to compare the different methods. Currently I only use the timings, but I could also work with either all of the data of the finished chains or with all the complete and partial data.

Things would of course be easier if I would just submit a huge job on a cluster, but, apart from not having access right now, doing things locally and in this way shortens the development iteration times.

@ahartikainen

(source → cpp?)

Just the stan functions/model blocks. See e.g. this Adjoint ODE Prototype - RFC - Please Test - #7 by Funko_Unko or this Adjoint ODE Prototype - RFC - Please Test - #16 by Funko_Unko monster of a figure for an example.