Issues with cmdstanpy on Apple Silicon and newest MacOS

@jbaranowski A new build specifically for the M1 macs should be available soon. If you get a chance to make a new conda environment and test it I’d really appreciate that.

It’s currently uploading to the anaconda servers, at which point it will take a little while to be available. You can see if it is by running:

 conda search -c conda-forge cmdstan --platform=osx-arm64

If you see a message like No match found for: cmdstan. then it needs more time

I have the same problem that @jbaranowski described, so I followed these steps:

  1. checked with mentioned “conda search” command for availability of build
  2. created new env
  3. installed cmdstanpy via conda

Creating model failed with output:

INFO:cmdstanpy:compiling stan file /Users/adrian/Documents/LAB4/tes.stan to exe file /Users/adrian/Documents/LAB4/tes
WARNING:cmdstanpy:CmdStan's precompiled header (PCH) files may need to be rebuilt.If your model failed to compile please run cmdstanpy.rebuild_cmdstan().
If the issue persists please open a bug report
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/Users/adrian/Documents/LAB4/testgp.ipynb Cell 4' in <cell line: 1>()
----> 1 model = CmdStanModel(stan_file='tes.stan')

File ~/opt/anaconda3/envs/cmdstan/lib/python3.10/site-packages/cmdstanpy/model.py:210, in CmdStanModel.__init__(self, model_name, stan_file, exe_file, compile, stanc_options, cpp_options, user_header)
    208 self.compile(force=str(compile).lower() == 'force')
    209 if self._exe_file is None:
--> 210     raise ValueError(
    211         'Unable to compile Stan model file: {}.'.format(
    212             self._stan_file
    213         )
    214     )

ValueError: Unable to compile Stan model file: /Users/adrian/Documents/LAB4/tes.stan.

As suggested I tried to rebuild, but it also failed:

Compiling:  (01:09) | ████▉      | make: ** ... stansummary] Error 1
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
File ~/opt/anaconda3/envs/cmdstan/lib/python3.10/site-packages/cmdstanpy/install_cmdstan.py:117, in build(verbose, progress, cores)
    116     progress_hook: Any = _wrap_build_progress_hook()
--> 117     do_command(cmd, fd_out=None, pbar=progress_hook)
    118 else:

File ~/opt/anaconda3/envs/cmdstan/lib/python3.10/site-packages/cmdstanpy/utils.py:1089, in do_command(cmd, cwd, fd_out, pbar)
   1086             msg = 'Command {}\n\t{} {}'.format(
   1087                 cmd, returncode_msg(proc.returncode), serror
   1088             )
-> 1089             raise RuntimeError(msg)
   1090 except OSError as e:

RuntimeError: Command ['make', 'build', '-j1']
	error during processing No such file or directory

During handling of the above exception, another exception occurred:

CmdStanInstallError                       Traceback (most recent call last)
/Users/adrian/Documents/LAB4/testgp.ipynb Cell 3' in <cell line: 2>()
      1 from cmdstanpy import rebuild_cmdstan
----> 2 rebuild_cmdstan()

File ~/opt/anaconda3/envs/cmdstan/lib/python3.10/site-packages/cmdstanpy/install_cmdstan.py:223, in rebuild_cmdstan(verbose, progress, cores)
    221     with pushd(cmdstan_path()):
    222         clean_all(verbose)
--> 223         build(verbose, progress, cores)
    224         compile_example(verbose)
    225 except ValueError as e:

File ~/opt/anaconda3/envs/cmdstan/lib/python3.10/site-packages/cmdstanpy/install_cmdstan.py:123, in build(verbose, progress, cores)
    119         do_command(cmd, fd_out=None)
    121 except RuntimeError as e:
    122     # pylint: disable=raise-missing-from
--> 123     raise CmdStanInstallError(f'Command "make build" failed\n{str(e)}')
    124 if not os.path.exists(os.path.join('bin', 'stansummary' + EXTENSION)):
    125     raise CmdStanInstallError(
    126         f'bin/stansummary{EXTENSION} not found'
    127         ', please rebuild or report a bug!'
    128     )

CmdStanInstallError: Command "make build" failed
Command ['make', 'build', '-j1']
	error during processing No such file or directory

So it seems that there is still an issue somewhere in conda version or maybe I am doing something wrong

Hi @adudek -

Can you check to make sure you are using the osx-arm64 specific version? The output of conda info would be helpful as well

I figured it out. It turned out, that I used VSCode, which wasn’t part of conda installation (just another independent installation). Even when connected to conda kernel it caused mentioned errors.

With VSCode lunched directly from conda (e.g. from Anaconda Navigator) all works fine :)

1 Like

Hello All.
I have an issue with the cmdstan installation on my mac m1 (python 3.11) after the “pip install” of cmdstanpy.

Run the below command in env

install_cmdstan
ls -F ~/.cmdstan

And getting below error:

Installing CmdStan version: 2.32.2
Install directory: /Users/reederey/.cmdstan
Downloading CmdStan version 2.32.2
Download successful, file: /var/folders/z5/89rvc9x90c740pxh34npyshw0000gn/T/tmp5m5ds1zl
Extracting distribution
Unpacked download as cmdstan-2.32.2
Building version cmdstan-2.32.2, may take several minutes, depending on your system.
Test model compilation
Traceback (most recent call last):
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 361, in compile_example
do_command(cmd, fd_out=None)
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/utils/command.py”, line 76, in do_command
raise RuntimeError(msg)
RuntimeError: Command [‘make’, ‘examples/bernoulli/bernoulli’]
error during processing No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/reederey/miniforge3/envs/ts_ml/bin/install_cmdstan”, line 8, in
sys.exit(main())
^^^^^^^^^^
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 668, in main
run_install(InstallationSettings(**args))
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 602, in run_install
install_version(
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 424, in install_version
compile_example(verbose)
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 364, in compile_example
raise CmdStanInstallError(f’Command “make clean-all” failed\n{e}')
cmdstanpy.install_cmdstan.CmdStanInstallError: Command “make clean-all” failed
Command [‘make’, ‘examples/bernoulli/bernoulli’]
error during processing No such file or directory

The error message I’ve received indicates that the command make examples/bernoulli/bernoulli is failing because it can’t find the required files.

Any help on below? Thanks.

Same after using below
from cmdstanpy import rebuild_cmdstan
rebuild_cmdstan()

Error:

name: CmdStanInstallError,
message: Command \make clean-all\ failed\nCommand [‘make’, ‘examples/bernoulli/bernoulli’]\n\terror during processing No such file or directory,
stack: \u001b[0;31m–

Do you have make installed (can you run it by itself on the command line)?

Hello. Thanks a lot for your answer. Yes, I have “make”.
//GNU Make 3.81
This program was built for i386-apple-darwin11.3.0//

I have tried only use the following command in the terminal:
install_cmdstan
ls -F ~/.cmdstan

from cmdstanpy import rebuild_cmdstan
rebuild_cmdstan() ---- only used in Python IDE.

No successful install with the following reference:

Blockquote
RuntimeError: Command [‘make’, ‘examples/bernoulli/bernoulli’]
error during processing No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/Users/reederey/miniforge3/envs/ts_ml/bin/install_cmdstan”, line 8, in
sys.exit(main())
^^^^^^^^^^
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 668, in main
run_install(InstallationSettings(**args))
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 602, in run_install
install_version(
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 424, in install_version
compile_example(verbose)
File “/Users/reederey/miniforge3/envs/ts_ml/lib/python3.11/site-packages/cmdstanpy/install_cmdstan.py”, line 364, in compile_example
raise CmdStanInstallError(f’Command “make clean-all” failed\n{e}')
cmdstanpy.install_cmdstan.CmdStanInstallError: Command “make clean-all” failed
Command [‘make’, ‘examples/bernoulli/bernoulli’]
error during processing No such file or directory


Good day, I’m still experiencing issue with installation of cmdstan. Please see attached screen.

Can you share the output of conda info and conda list?

Thanks for your answer and williness to help. Pls find below.

    active environment : ts_ml
    active env location : /Users/reederey/miniforge3/envs/ts_ml
            shell level : 2
       user config file : /Users/reederey/.condarc
 populated config files : /Users/reederey/miniforge3/.condarc
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.5=0
                          __unix=0=0
       base environment : /Users/reederey/miniforge3  (writable)
      conda av data dir : /Users/reederey/miniforge3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/reederey/miniforge3/pkgs
                          /Users/reederey/.conda/pkgs
       envs directories : /Users/reederey/miniforge3/envs
                          /Users/reederey/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.3.1 requests/2.28.2 CPython/3.10.10 Darwin/22.6.0 OSX/13.5
                UID:GID : 501:20
             netrc file : /Users/reederey/.netrc
           offline mode : False
# packages in environment at /Users/reederey/miniforge3/envs/ts_ml:
#
# Name                    Version                   Build  Channel
annotated-types           0.5.0                    pypi_0    pypi
anyio                     3.7.1                    pypi_0    pypi
appnope                   0.1.3                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.2.3                    pypi_0    pypi
asttokens                 2.2.1                    pypi_0    pypi
async-lru                 2.0.4                    pypi_0    pypi
attrs                     23.1.0                   pypi_0    pypi
babel                     2.12.1                   pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.12.2                   pypi_0    pypi
bleach                    6.0.0                    pypi_0    pypi
blis                      0.9.1                    pypi_0    pypi
build                     0.10.0                   pypi_0    pypi
bzip2                     1.0.8                h3422bc3_4    conda-forge
ca-certificates           2023.5.7             hf0a4a13_0    conda-forge
cachecontrol              0.13.1                   pypi_0    pypi
catalogue                 2.0.9                    pypi_0    pypi
certifi                   2023.7.22                pypi_0    pypi
cffi                      1.15.1                   pypi_0    pypi
charset-normalizer        3.2.0                    pypi_0    pypi
cleo                      2.0.1                    pypi_0    pypi
click                     8.1.6                    pypi_0    pypi
cloudpickle               2.2.1                    pypi_0    pypi
cmdstanpy                 1.1.0                    pypi_0    pypi
comm                      0.1.4                    pypi_0    pypi
confection                0.1.1                    pypi_0    pypi
contourpy                 1.1.0                    pypi_0    pypi
convertdate               2.4.0                    pypi_0    pypi
crashtest                 0.4.1                    pypi_0    pypi
cvxpy                     1.3.2                    pypi_0    pypi
cycler                    0.11.0                   pypi_0    pypi
cymem                     2.0.7                    pypi_0    pypi
cython                    3.0.0                    pypi_0    pypi
dask                      2023.8.0                 pypi_0    pypi
debugpy                   1.6.7.post1              pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
deprecated                1.2.14                   pypi_0    pypi
dill                      0.3.7                    pypi_0    pypi
distlib                   0.3.7                    pypi_0    pypi
distributed               2023.8.0                 pypi_0    pypi
dulwich                   0.21.5                   pypi_0    pypi
ecos                      2.0.12                   pypi_0    pypi
ephem                     4.1.4                    pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
executing                 1.2.0                    pypi_0    pypi
fastai                    2.7.12                   pypi_0    pypi
fastcore                  1.5.29                   pypi_0    pypi
fastdownload              0.0.7                    pypi_0    pypi
fastjsonschema            2.18.0                   pypi_0    pypi
fastprogress              1.0.3                    pypi_0    pypi
filelock                  3.12.2                   pypi_0    pypi
fonttools                 4.42.0                   pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
fsspec                    2023.6.0                 pypi_0    pypi
greykite                  0.5.1                    pypi_0    pypi
hijri-converter           2.3.1                    pypi_0    pypi
holidays                  0.30                     pypi_0    pypi
holidays-ext              0.0.8                    pypi_0    pypi
html5lib                  1.1                      pypi_0    pypi
idna                      3.4                      pypi_0    pypi
imbalanced-learn          0.11.0                   pypi_0    pypi
importlib-metadata        6.8.0                    pypi_0    pypi
iniconfig                 2.0.0                    pypi_0    pypi
installer                 0.7.0                    pypi_0    pypi
ipykernel                 6.25.1                   pypi_0    pypi
ipython                   8.14.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                8.1.0                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jaraco-classes            3.3.0                    pypi_0    pypi
jedi                      0.19.0                   pypi_0    pypi
jinja2                    3.1.2                    pypi_0    pypi
joblib                    1.3.2                    pypi_0    pypi
json5                     0.9.14                   pypi_0    pypi
jsonpointer               2.4                      pypi_0    pypi
jsonschema                4.19.0                   pypi_0    pypi
jsonschema-specifications 2023.7.1                 pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            8.3.0                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.3.1                    pypi_0    pypi
jupyter-events            0.7.0                    pypi_0    pypi
jupyter-lsp               2.2.0                    pypi_0    pypi
jupyter-server            2.7.1                    pypi_0    pypi
jupyter-server-terminals  0.4.4                    pypi_0    pypi
jupyterlab                4.0.5                    pypi_0    pypi
jupyterlab-pygments       0.2.2                    pypi_0    pypi
jupyterlab-server         2.24.0                   pypi_0    pypi
jupyterlab-widgets        3.0.8                    pypi_0    pypi
keyring                   24.2.0                   pypi_0    pypi
kiwisolver                1.4.4                    pypi_0    pypi
korean-lunar-calendar     0.3.1                    pypi_0    pypi
langcodes                 3.3.0                    pypi_0    pypi
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libsqlite                 3.42.0               hb31c410_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
lightgbm                  4.0.0                    pypi_0    pypi
llvmlite                  0.40.1                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
lockfile                  0.12.2                   pypi_0    pypi
lunarcalendar-ext         0.0.1                    pypi_0    pypi
markupsafe                2.1.3                    pypi_0    pypi
matplotlib                3.7.2                    pypi_0    pypi
matplotlib-inline         0.1.6                    pypi_0    pypi
mistune                   3.0.1                    pypi_0    pypi
more-itertools            10.1.0                   pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
msgpack                   1.0.5                    pypi_0    pypi
murmurhash                1.0.9                    pypi_0    pypi
nbclassic                 1.0.0                    pypi_0    pypi
nbclient                  0.8.0                    pypi_0    pypi
nbconvert                 7.7.4                    pypi_0    pypi
nbformat                  5.9.2                    pypi_0    pypi
ncurses                   6.4                  h7ea286d_0    conda-forge
nest-asyncio              1.5.7                    pypi_0    pypi
networkx                  3.1                      pypi_0    pypi
notebook                  7.0.2                    pypi_0    pypi
notebook-shim             0.2.3                    pypi_0    pypi
numba                     0.57.1                   pypi_0    pypi
numpy                     1.25.2                   pypi_0    pypi
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   3.1.1                h53f4e23_1    conda-forge
osqp                      0.6.3                    pypi_0    pypi
overrides                 7.4.0                    pypi_0    pypi
packaging                 23.1                     pypi_0    pypi
pandas                    2.0.3                    pypi_0    pypi
pandocfilters             1.5.0                    pypi_0    pypi
parso                     0.8.3                    pypi_0    pypi
partd                     1.4.0                    pypi_0    pypi
pathy                     0.10.2                   pypi_0    pypi
patsy                     0.5.3                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    10.0.0                   pypi_0    pypi
pip                       23.2.1                   pypi_0    pypi
pip-review                1.3.0                    pypi_0    pypi
pkginfo                   1.9.6                    pypi_0    pypi
platformdirs              3.10.0                   pypi_0    pypi
plotly                    5.16.1                   pypi_0    pypi
pluggy                    1.2.0                    pypi_0    pypi
pmdarima                  2.0.3                    pypi_0    pypi
poetry                    1.5.1                    pypi_0    pypi
poetry-core               1.6.1                    pypi_0    pypi
poetry-plugin-export      1.4.0                    pypi_0    pypi
preshed                   3.0.8                    pypi_0    pypi
prometheus-client         0.17.1                   pypi_0    pypi
prompt-toolkit            3.0.39                   pypi_0    pypi
psutil                    5.9.5                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.2                    pypi_0    pypi
pyarrow                   12.0.1                   pypi_0    pypi
pycparser                 2.21                     pypi_0    pypi
pydantic                  2.2.0                    pypi_0    pypi
pydantic-core             2.6.0                    pypi_0    pypi
pygments                  2.16.1                   pypi_0    pypi
pymeeus                   0.5.12                   pypi_0    pypi
pyod                      1.1.0                    pypi_0    pypi
pyparsing                 3.1.1                    pypi_0    pypi
pyproject-hooks           1.0.0                    pypi_0    pypi
pytest                    7.4.0                    pypi_0    pypi
pytest-runner             6.0.0                    pypi_0    pypi
python                    3.11.4          h47c9636_0_cpython    conda-forge
python-dateutil           2.8.2                    pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
pyts                      0.13.0                   pypi_0    pypi
pytz                      2023.3                   pypi_0    pypi
pyyaml                    6.0.1                    pypi_0    pypi
pyzmq                     25.1.1                   pypi_0    pypi
qdldl                     0.1.7.post0              pypi_0    pypi
qtconsole                 5.4.3                    pypi_0    pypi
qtpy                      2.3.1                    pypi_0    pypi
rapidfuzz                 3.2.0                    pypi_0    pypi
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.30.2                   pypi_0    pypi
requests                  2.31.0                   pypi_0    pypi
requests-toolbelt         1.0.0                    pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rpds-py                   0.9.2                    pypi_0    pypi
ruptures                  1.1.8                    pypi_0    pypi
scikit-base               0.5.1                    pypi_0    pypi
scikit-learn              1.3.0                    pypi_0    pypi
scipy                     1.11.1                   pypi_0    pypi
scs                       3.2.3                    pypi_0    pypi
seaborn                   0.12.2                   pypi_0    pypi
send2trash                1.8.2                    pypi_0    pypi
setuptools                68.1.0                   pypi_0    pypi
shellingham               1.5.3                    pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sktime                    0.21.0                   pypi_0    pypi
smart-open                6.3.0                    pypi_0    pypi
sniffio                   1.3.0                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
soupsieve                 2.4.1                    pypi_0    pypi
spacy                     3.6.1                    pypi_0    pypi
spacy-legacy              3.0.12                   pypi_0    pypi
spacy-loggers             1.0.4                    pypi_0    pypi
srsly                     2.4.7                    pypi_0    pypi
stack-data                0.6.2                    pypi_0    pypi
statsmodels               0.14.0                   pypi_0    pypi
stumpy                    1.11.1                   pypi_0    pypi
sympy                     1.12                     pypi_0    pypi
tblib                     2.0.0                    pypi_0    pypi
tenacity                  8.2.3                    pypi_0    pypi
terminado                 0.17.1                   pypi_0    pypi
testfixtures              7.1.0                    pypi_0    pypi
thinc                     8.2.0                    pypi_0    pypi
threadpoolctl             3.2.0                    pypi_0    pypi
tinycss2                  1.2.1                    pypi_0    pypi
tk                        8.6.12               he1e0b03_0    conda-forge
tomlkit                   0.12.1                   pypi_0    pypi
toolz                     0.12.0                   pypi_0    pypi
torch                     2.0.1                    pypi_0    pypi
torchvision               0.15.2                   pypi_0    pypi
tornado                   6.3.3                    pypi_0    pypi
tqdm                      4.66.1                   pypi_0    pypi
traitlets                 5.9.0                    pypi_0    pypi
trove-classifiers         2023.8.7                 pypi_0    pypi
tsai                      0.3.7                    pypi_0    pypi
tsfresh                   0.20.1                   pypi_0    pypi
tslearn                   0.6.1                    pypi_0    pypi
typer                     0.9.0                    pypi_0    pypi
typing-extensions         4.7.1                    pypi_0    pypi
tzdata                    2023.3                   pypi_0    pypi
uri-template              1.3.0                    pypi_0    pypi
urllib3                   2.0.4                    pypi_0    pypi
virtualenv                20.24.3                  pypi_0    pypi
wasabi                    1.1.2                    pypi_0    pypi
wcwidth                   0.2.6                    pypi_0    pypi
webcolors                 1.13                     pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.6.1                    pypi_0    pypi
wheel                     0.41.1                   pypi_0    pypi
widgetsnbextension        4.0.8                    pypi_0    pypi
wrapt                     1.15.0                   pypi_0    pypi
xattr                     0.10.1                   pypi_0    pypi
xgboost                   1.7.6                    pypi_0    pypi
xz                        5.2.6                h57fd34a_0    conda-forge
zict                      3.0.0                    pypi_0    pypi
zipp                      3.16.2                   pypi_0    pypi

It looks like you are not using cmdstanpy/cmdstan from conda-forge. Mixing conda environments and tools from brew/xcode seems to cause headaches on recent Macs