Frustrating problems with Pystan (anaconda Python)

I did actually upgrade my pip before attempting to install pystan (I am using pip 21.2.4). It should be latest pip version, but I still got pystan 2.19 on Mac. It was working fine with Ubuntu though.

It may be conda then. PyStan isn’t tested on conda at all.

I’d suggest trying to get things working using the global environment or creating a virtualenv.

You’re the first person who has had this precise experience, I think. If someone else encounters this, I think we would probably want to add an FAQ item mentioning that things are unlikely to work under conda.

I would not recommend using global python on macOS.

Also, if you have conda, how do you call your python and how did you call pip? Make sure that you really have updated your pip and the pip you use is actually the correct pip for your python.

I did use virtualenv.

Yes, my pip is updated, using python3 -m pip install pip -U. I have triple checked about my pip version.

As for conda, I called my script from inside conda environment with python3 script-name.py

Let me summarize it again, so that it does not get mixed up. Here are my problems:

  1. I was not able to install pystan 3.x on Mac, I was using pip 21.2.4 on Python 3.9.2. I still got pystan 2.19 installed no matter how often I repeatedly tried to update my pip, and specifying python3 -m pip install pystan==3.2.0 did not work too. I got a message ERROR: Could not find a version that satisfies the requirement httpstan<4.6,>=4.5 (from pystan)

  2. I then tried on 3 different Linux machines, all either on Python 3.8 or 3.9. I was able to install pystan 3.1.1 or 3.2.0. But then I got 3 different errors on these different machines (see above replies). I tried on both virtualenv (using pip) and conda environment. I tried the example code from the website as well. It worked in one machine, it did not in another one. What confused me more, I have another script that was running fine before suddenly did not work, and I got one of those errors I have posted above. I don’t think there is anything wrong with my machines specs, and I do not think the error came from my model/script as well. I had no idea what the error messages mean, and if there is no quick fix to these, I might as well move on with another framework.

Is that macOS M1 machine?

I am not too sure about that, but I assume no. It is Macbook pro retina 13" late 2013, intel i5 2.4 GHz, 8GB DDR3 RAM

Ok, not M1.

What OS version do you have? 10.14?

I think we don’t have wheels for it, only for 10.15 (@ariddell ?)

How do you run these Linux machines? Are they native or in VM? If VM, there might be some memory setting that is overflowing and causes the script to crash.

It might be easiest if we dealt with each platform separately.

If you cannot get things working on a supported platform (see https://pystan.readthedocs.io/en/latest/installation.html for requirements) then you should open a bug report at https://github.com/stan-dev/pystan/issues. Might make sense to start with Linux since it’s very likely we can quickly resolve the issue.

Conda is not supported right now so let’s leave that platform/distribution to one side.

The macOS is actually 10.14. Do I have to update it to 10.15 to use pystan 3.x on Mac then?
I did not find any information related to macOS version requirements in the documentation.

As for the Ubuntu machines, they are all in VMs. I just realized that the OS are either 16.04 or 18.04. Any chance the issues were because of the OS version as well? But then, it did not answer my question, why I used to be able to run my scripts without error previously under the same OS? If it is memory setting, how can I find out the way to solve the issue? If these issues are due to the slight variations of the OS versions/machine specs, is there anywhere in the documentation about detailed specs required to run pystan smoothly?

Neither 16.04 nor 18.04 are supported. So that solves that problem.

I don’t really understand macOS wheel compatibility. We rely on GitHub Actions and the multibuild project for building the macOS wheels. We set a variable MACOSX_DEPLOYMENT_TARGET: "10.9" – but perhaps this is being ignored, or is impossible to satisfy for some reason.

Wish I could be of more assistance.

Hi, I think conda has a version that might work.

Can you try the following (probably a good idea to use a new conda environment which is created with conda-forge flag)

conda create -n stan -c conda-forge python=3.9
conda activate stan # or source activate stan


conda install pystan httpstan -c conda-forge

I have the same issue. I have macOS 11.6 and ran the following command:

conda create -n stan_env -c conda-forge python=3.9 numpy cython pandas matplotlib scipy pystan jupyter notebook

Once installed, I needed to add these commands at the top of my jupyter notebook (in order to avoid the error RuntimeError: asyncio.run() cannot be called from a running event loop):

import nest_asyncio
nest_asyncio.apply()
del nest_asyncio

Finally, running the quick start code from the doc (PyStan — pystan 3.3.0 documentation), I get the same error:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/myuser/opt/miniconda3/envs/stan_env/lib/python3.9/site-packages/httpstan/stanc'

and at the bottom of the error stream:

ValueError: The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc.

I tried a bunch of different variations of the conda install but nothing works.

As PyStan 3 does not publish conda packages, conda isn’t officially supported. Using a normal virtualenv should work fine for you on macOS 11.6.

If someone has had success using conda with PyStan 3, perhaps they could post a quick tutorial (as a separate post) describing the required steps.

1 Like

Perhaps some of these "how do i even install the dependencies to run the software as a new user on platform x using package managers y and z " problems could be avoided by distributing a self-contained and tested Stan container image with pinned versions of dependencies (compilers, libraries, etc) atop a supported base image (e.g. debian stable).

I see earlier this year @wm1995 shared an example Dockerfile that installs pystan3 atop a python:latest image, which appears to be based on debian 11 bullseye, which is one of the operating systems supported by pystan:

With a container image there would still be a bit of a learning barrier for new users unfamiliar with wrangling container images: e.g. how do i run a container image locally (in this case installing docker for mac or perhaps an alternative like podman), how do i configure volume mounts and port forwarding to be able to share data and source code with a container or view a notebook hosted in the container – but there would no longer be any opportunity for users to accidentally pick combinations of build environments and package managers that aren’t supported or don’t work.

1 Like

I’m cautiously optimistic that all these problems will disappear over the next 24 months as people stop using Linux distributions published before 2019.

I just want to echo the above, I’ve tried pip install pystan from two different virtual environments (virtualenv and pipenv) on MacOs 10.14 with two different versions of Python (3.8.8 and 3.9.5) and in every case, with two different pip versions (20.2.3 and 21.3), none of them are able to install pystan 3.x, every time it falls back to 2.19.

This should not happen on intel macOS machines. (M1 is not supported right now.) Can you describe your environment in greater detail?

(That said, if you’re not using anaconda, you might want to create a new post as this one concerns Anaconda.)

Hi, I hope you are well - do you have an idea when pystan 3 could be available for the M1 Macs?

Thank you so much for all the work on it

Hi @Replicant_Rin - there was recently this post here:

2 Likes

Hi @WardBrian

Thank you so much! Had to create a new virtual env but it did work (after failing twice lol) !

1 Like