Any way to retrieve the best fitting parameters for a failed optimizing?

General
1

I use pystan 2.18.0 and would like to optimizing a model over different subset of a data set. However sometimes the optimizer complains:

Optimization terminated with error:
Line search failed to achieve a sufficient decrease, no more progress can be made

When this happens, pystan raises a RuntimeError and does not return fitted parameters.
My current workaround is ignore the error, but I found it would leave “holes” in the subset map.
My problem is, is there a way to retrieve the fitted parameters. even if it does not reach the criterion for convergence? Sometimes the function just get too “flat” or the tol_param just set to be too small. Either case, the best-fitted parameters may be still of use.

2

Ok, now I improved my program to catch the error, set loose the ‘tol_param’ parameter(by making it larger) and retry. Mostly it would only have to retry once or twice. Still it’s not a very elegant solution as it re-evaluates the same optimization procedure, but it’s acceptable in my scenario.

3

This may be a bit of a detour, but if you are willing to use sampling instead of optimization (which is probably a good idea on its own if that is failing frequently), and use CmdStan instead of PyStan you will have the samples from the run(s) written to disk as they are being sampled and you will be able to access intermediate states. This would likely not be a full analysis, but neither would and intermediate optimization state, and it may serve the purpose you are seeking.

As a frequent Python user I initially resisted switching to a command line, but I realized that using PyStan is essentially creating a dictionary with data for the Stan model. With the most recent (development branch) CmdStan version you can use json files, which makes it straightforward to export Python dictionaries and pass them as arguments to the compiled Stan model (I had never used json for anything useful before). Reading the output back to Python may or may not require a couple more steps to get the same output given by PyStan, but I found it to be worth the additional control CmdStan allows.

4

Try ArviZ (supports also PyStan) to read CmdStan csv to Python

import arviz as az
inference_data = az.from_cmdstan(posterior="path/to/output[0-9].csv") # you can use glob string or a list of paths

https://arviz-devs.github.io/arviz/generated/arviz.from_cmdstan.html#arviz.from_cmdstan

2 Likes