Problems with installing Pystan in Python 3.10 on Ubuntu 20.04 LTS

Interfaces PyStan
1

I have troubles installing Pystan in Python 3.10 on my Ubuntu 20.04 system. I have already tried all the possible troubleshooting, as well as e.g. a similar issue (Error installing Pystan in python 3.10 with gcc 9.2.0).

This is the error message I receive when I run the following command: sudo pip install pystan

Building wheels for collected packages: pysimdjson
  Building wheel for pysimdjson (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      /home/anvlo/PycharmProjects/PhD/venv/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      /home/anvlo/PycharmProjects/PhD/venv/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/simdjson
      copying simdjson/__init__.py -> build/lib.linux-x86_64-cpython-310/simdjson
      running build_ext
      building 'csimdjson' extension
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/simdjson
      x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-install-1dmgbq2u/pysimdjson_aa6a3887b90247e2bca76445194b5d1f/.eggs/pybind11-2.10.0-py3.10.egg/pybind11/include -I/home/anvlo/PycharmProjects/PhD/venv/include -I/usr/include/python3.10 -c simdjson/binding.cpp -o build/temp.linux-x86_64-cpython-310/simdjson/binding.o -std=c++11
      In file included from /tmp/pip-install-1dmgbq2u/pysimdjson_aa6a3887b90247e2bca76445194b5d1f/.eggs/pybind11-2.10.0-py3.10.egg/pybind11/include/pybind11/detail/../attr.h:13,
                       from /tmp/pip-install-1dmgbq2u/pysimdjson_aa6a3887b90247e2bca76445194b5d1f/.eggs/pybind11-2.10.0-py3.10.egg/pybind11/include/pybind11/detail/class.h:12,
                       from /tmp/pip-install-1dmgbq2u/pysimdjson_aa6a3887b90247e2bca76445194b5d1f/.eggs/pybind11-2.10.0-py3.10.egg/pybind11/include/pybind11/pybind11.h:13,
                       from simdjson/binding.cpp:7:
      /tmp/pip-install-1dmgbq2u/pysimdjson_aa6a3887b90247e2bca76445194b5d1f/.eggs/pybind11-2.10.0-py3.10.egg/pybind11/include/pybind11/detail/../detail/common.h:212:10: fatal error: Python.h: No such file or directory
        212 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pysimdjson
  Running setup.py clean for pysimdjson
Failed to build pysimdjson
Installing collected packages: pysimdjson, clikit, httpstan, pystan
  Attempting uninstall: pysimdjson
    Found existing installation: pysimdjson 5.0.2
    Uninstalling pysimdjson-5.0.2:
      Successfully uninstalled pysimdjson-5.0.2
  Running setup.py install for pysimdjson ... error

I have already tried to install Cmake, I have updated pip, setuptools, tried downloading pystan in python 3.9, all render the same error.

Thanks in advance for the help!

2

You need to compile httpstan from source and after it has been installed you can install pystan.

Do you have C++ toolchain installed?

3

Thanks for the fast reply! Could you give little more information on how to proceed? I have installed httpstan (python3 -m pip install httpstan) or is this not how to install it from source? I googled for C++ toolchain but no concrete instructions popped up.

4

C++ toolchain

sudo apt-get update
sudo apt-get install build-essential
gcc --version

Then follow Installation — httpstan 4.8.2 documentation

5

C++ toolchain installment worked. It feels like with every error I resolve, I get 10 new ones:

python3 -m pip install poetry make , worked

But soon I get the following errors:

anvlo@bystre21:~$ python3 -m poetry build
cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

anvlo@bystre21:~$ python3 -m pip install dist/*.whl
Defaulting to user installation because normal site-packages is not writeable
WARNING: Requirement 'dist/*.whl' looks like a filename, but the file does not exist
ERROR: *.whl is not a valid wheel filename.
6

Hi,

You are not far from ready.

Ok that certainly is an error

In python 3.10 is should be collections.abc.Mapping.

I wonder what library causes that error. Was there a longer exception message? I did not find that part in our codebase.

cc @ariddell

7

No, everything that popped up I put here. I have tried to resolve the problem but no luck so far.

8

I think this works on Ubuntu 20.04.

sudo apt install python3-pip
python3 -m pip install -U pip
python3 -m pip install pystan

You do not need to install httpstan from source.

Reading my own notes here: Ubuntu 20.04 cannot install httpstan's manylinux_2_24_x86_64 wheel ("No matching distribution found") · Issue #323 · stan-dev/pystan · GitHub

Update: Just verified this. Everything works on Ubuntu 20.04. httpstan-4.8.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl downloaded and installed, as it should.

9

I’ve tried this just now but still the same, original error is given when I try the commands. Thanks again for all your time and effort…

10

The problem is with the python 3.10.

Could you try with 3.9?

We need to figure out where is that error coming from.

11

I have no problems with Python 3.10 on Ubuntu 20.04 LTS. I’m using the deadsnakes ppa to get Python 3.10. Index of /deadsnakes/ppa/ubuntu

12

Looks like the issue might be with pysimdjson. For some reason I end up with pysimdjson==3.2.0 when I pip install pystan on Ubuntu 20.04. @avdvloet , I’m not sure why your system is trying to install a different version.

1 Like
13

I succeeded in installing Pystan without any problems when I created a venv where I downgraded to Python 3.9. For this I used the following commands from @ariddell :

sudo apt install python3-pip
python3 -m pip install -U pip
python3 -m pip install pystan

For now there is no real need for me to work in Python 3.10, 3.9 will work just fine. Thank you again for all the help and feedback!

2 Likes
14

This error seems to not be solved yet. I have a similar error in Python 3.11.0 in WSL (Ubuntu 22.04.1 LTS)

$ pip install pystan
Collecting pystan
  Using cached pystan-3.6.0-py3-none-any.whl (13 kB)
Collecting aiohttp<4.0,>=3.6
  Using cached aiohttp-3.8.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
Collecting clikit<0.7,>=0.6
  Using cached clikit-0.6.2-py2.py3-none-any.whl (91 kB)
Collecting httpstan<4.10,>=4.9
  Using cached httpstan-4.9.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.4 MB)
Requirement already satisfied: numpy<2.0,>=1.19 in /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages (from pystan) (1.24.1)
Collecting pysimdjson<6.0.0,>=5.0.2
  Using cached pysimdjson-5.0.2.tar.gz (547 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: setuptools in /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages (from pystan) (66.0.0)
Collecting attrs>=17.3.0
  Using cached attrs-22.2.0-py3-none-any.whl (60 kB)
Collecting charset-normalizer<3.0,>=2.0
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting multidict<7.0,>=4.5
  Using cached multidict-6.0.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (117 kB)
Collecting async-timeout<5.0,>=4.0.0a3
  Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting yarl<2.0,>=1.0
  Using cached yarl-1.8.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (278 kB)
Collecting frozenlist>=1.1.1
  Using cached frozenlist-1.3.3-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (154 kB)
Collecting aiosignal>=1.1.2
  Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting crashtest<0.4.0,>=0.3.0
  Using cached crashtest-0.3.1-py3-none-any.whl (7.0 kB)
Collecting pastel<0.3.0,>=0.2.0
  Using cached pastel-0.2.1-py2.py3-none-any.whl (6.0 kB)
Requirement already satisfied: pylev<2.0,>=1.3 in /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages (from clikit<0.7,>=0.6->pystan) (1.4.0)
Requirement already satisfied: appdirs<2.0,>=1.4 in /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages (from httpstan<4.10,>=4.9->pystan) (1.4.4)
Collecting marshmallow<4.0,>=3.10
  Using cached marshmallow-3.19.0-py3-none-any.whl (49 kB)
Collecting webargs<9.0,>=8.0
  Using cached webargs-8.2.0-py3-none-any.whl (30 kB)
Requirement already satisfied: packaging>=17.0 in /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages (from marshmallow<4.0,>=3.10->httpstan<4.10,>=4.9->pystan) (23.0)
Collecting idna>=2.0
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Building wheels for collected packages: pysimdjson
  Building wheel for pysimdjson (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/simdjson
      copying simdjson/__init__.py -> build/lib.linux-x86_64-cpython-311/simdjson
      copying simdjson/__init__.pyi -> build/lib.linux-x86_64-cpython-311/simdjson
      copying simdjson/py.typed -> build/lib.linux-x86_64-cpython-311/simdjson
      running build_ext
      building 'csimdjson' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/simdjson
      /home/homero/software/miniconda3/envs/replay/bin/x86_64-conda-linux-gnu-cc -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/homero/software/miniconda3/envs/replay/include -fPIC -O2 -isystem /home/homero/software/miniconda3/envs/replay/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/homero/software/miniconda3/envs/replay/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/homero/software/miniconda3/envs/replay/include -fPIC -I/home/homero/software/miniconda3/envs/replay/include/python3.11 -c simdjson/csimdjson.cpp -o build/temp.linux-x86_64-cpython-311/simdjson/csimdjson.o
      error: command '/home/homero/software/miniconda3/envs/replay/bin/x86_64-conda-linux-gnu-cc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pysimdjson
  Running setup.py clean for pysimdjson
Failed to build pysimdjson
Installing collected packages: pysimdjson, pastel, multidict, marshmallow, idna, frozenlist, crashtest, charset-normalizer, attrs, async-timeout, yarl, webargs, clikit, aiosignal, aiohttp, httpstan, pystan
  Running setup.py install for pysimdjson ... error
  error: subprocess-exited-with-error

  × Running setup.py install for pysimdjson did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running install
      /home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/simdjson
      copying simdjson/__init__.py -> build/lib.linux-x86_64-cpython-311/simdjson
      copying simdjson/__init__.pyi -> build/lib.linux-x86_64-cpython-311/simdjson
      copying simdjson/py.typed -> build/lib.linux-x86_64-cpython-311/simdjson
      running build_ext
      building 'csimdjson' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/simdjson
      /home/homero/software/miniconda3/envs/replay/bin/x86_64-conda-linux-gnu-cc -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/homero/software/miniconda3/envs/replay/include -fPIC -O2 -isystem /home/homero/software/miniconda3/envs/replay/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/homero/software/miniconda3/envs/replay/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/homero/software/miniconda3/envs/replay/include -fPIC -I/home/homero/software/miniconda3/envs/replay/include/python3.11 -c simdjson/csimdjson.cpp -o build/temp.linux-x86_64-cpython-311/simdjson/csimdjson.o
      error: command '/home/homero/software/miniconda3/envs/replay/bin/x86_64-conda-linux-gnu-cc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pysimdjson

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
15

Ok, I’m not sure what happened. I installed pystan using conda install -c conda-forge pystan. Then I was still having an error when running a simple stan code linregstan.py (841 Bytes):

FileNotFoundError: [Errno 2] No such file or directory: PosixPath(‘/home/homero/software/miniconda3/envs/replay/lib/python3.11/site-packages/httpstan/stanc’)

But then I did pip install pystan. The installation finished successfully and I no longer have any error.

16

I think there is missing wheel in pysimdjson for python 3.11

And conda has one for python 3.11

https://anaconda.org/conda-forge/pysimdjson/files

So fix before pypi is updated is to install pysimdjson manually from conda(if possible)

conda install pysimdjson -c conda-forge
17

The missing python3.11 wheel in pypi is still an issue. (I solved it by using python3.9 for now)

18

i’ve managed to install it on Air M2 for my daughter studying her freshman year Theoretical Physics like this

1 Like
19

your notebook is lovely - just to note that you’re using CmdStanPy - which was developed, in part, to make it easy to download and install Stan.