I have been encountering a lot of cases where my PyStan code crashes on Mac OS Catalina, spawning a ‘ReportCrash’ and printing out the following repeatedly: libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Couldn’t close file libc++abi.dylib: terminating with uncau…

Finally traced back the issue to the matplotlib library and think I have fixed it for good by editing some code on their end, will raise an issue on their project to try and get this fixed as I presume this is affecting any package that leverages multiprocessing on Mac OS Catalina.

Ah I see, so calling build on each iteration should not affect my performance, thank you for the quick response. Do you happen to know how I can emulate return fit.extract(permuted=True) behaviour?

You don’t want to permute your samples. Just use all if your samples when doing further work. ArviZ also supports pystan3, read the docstring for adding the model to from_pystan. InferenceData can be saved to netCDF format. https://arviz-devs.github.io/arviz/generated/arviz.from_pystan.html#arviz.…

The reason I ask is to avoid having to change my entire pipeline, but before I go further I think I should maybe give more context on what I am trying to do. I’d like to run the equivalent of the following, though n_jobs does not exist in PyStan3 so I am not sure how to setup sample to run across mu…

PyStan3 uses threading to parallize over multiple cores. So it is already doing multiple cores (I think) If you want just dictionary try fit.to_frame().to_dict() or just use the DataFrame. Edit. I think to control num treads use (I need to verify this) import os os.environ["STAN_NUM_THREADS"] =…

Thank you, I have managed to get the fit integrated correctly. I am having some pretty serious performance issues though; I don’t seem to get any console output using build, is that intentional? It is making it hard to verify whether multiple cores are being utilised or not, as in activity monitor t…

@hwilde (or others), is it possible to get some more information about this fix? It’s making things very difficult for some runs on my machine.

Can you try to run pystan in virtual environment without matplotlib installed?

Didn’t even need to do that – just not importing/calling matplotlib seems to be enough to “fix” the problem. But this is not a good fix, since I really need to plot my results…

Can you import matplotlib after your sampling is done? This of course does not help for notebook work, but if you use scripts, then it should. See the issue here https://github.com/matplotlib/matplotlib/issues/15410

Ok, thanks. But is there in fact a matplotlib fix that can be applied? The problems affect plotting there in addition to Stan issues.

Issues with PyStan, multiprocessing and subsequently the PyStan3 pre-release on Mac OS Catalina

Developers

ariddell August 18, 2020, 12:31pm 21

You can also simply use an earlier version of Jupyter. I’ve posted instructions here: Using PyStan 3 with Jupyter Notebook and Jupyterlab

Topic		Replies	Views
Trouble with PyStan 3 and python multiprocessing PyStan pystan	9	2480	October 27, 2021
PyStan throws error when running chains in parallel (n_jobs > 1) PyStan pystan	8	3075	November 13, 2020
Unable to get PyStan working on a Mac PyStan fitting-issues	1	1644	December 1, 2020
New to Pystan, Always get this error when attempting to sample: ModuleNotFoundError: No module named 'stanfit4anon_model...' Modeling pystan	9	5463	February 9, 2022
Pystan on multiprocessing Modeling specification	14	4038	December 3, 2018

Issues with PyStan, multiprocessing and subsequently the PyStan3 pre-release on Mac OS Catalina

Related topics