Pystan causes crash (gcc compiler?), conda on Mac OS X

[pbadel]

Mac OS X 10.12.5, miniconda, python 3.6.8, pystan 2.19.0.0. I don’t know what Compiler/Toolkit I am using.

I have installed conda in my personal directories: python2 in /Users/peter/miniconda2 and python3 in /Users/peter/miniconda3 . My paths are set up such that “python” gives me /Users/peter/miniconda2/bin/python and “python3” gives me /Users/peter/miniconda3/bin/python . I get the same error below whether I use python2 or python3.

I am trying to run the “Getting started, Example 1” example at https://pystan.readthedocs.io/en/latest/getting_started.html (except I have added “verbose=True” to the line which gives the error), at the line

sm = pystan.StanModel(model_code=schools_code, verbose=True)

I get a long output and eventually the “too many errors” error. So I will just give the log to the first two errors.

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_19a09b474d1901f191444eaf8a6b8ce2 NOW.
INFO:pystan:OS: darwin, Python: 3.6.8 |Anaconda, Inc.| (default, Dec 29 2018, 19:04:46) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)], Cython 0.28.5
Compiling /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.pyx because it changed.
[1/1] Cythonizing /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.pyx
building 'stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416' extension
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/peter/miniconda3/include -arch x86_64 -I/Users/peter/miniconda3/include -arch x86_64 -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3 -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/src -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/eigen_3.3.3 -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0 -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include -I/Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include -I/Users/peter/miniconda3/include/python3.6m -c /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp -o /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.o -O2 -ftemplate-depth-256 -Wno-unused-function -Wno-uninitialized -std=c++1y
In file included from /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp:642:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:
/Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
 ^
In file included from /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp:646:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan_fit.hpp:12:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/posix_time/posix_time_types.hpp:10:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/time_clock.hpp:17:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/shared_ptr.hpp:17:
/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/smart_ptr/shared_ptr.hpp:370:174: error: no member named 'move' in namespace 'std'
    BOOST_CONSTEXPR shared_ptr( boost::detail::sp_internal_constructor_tag, element_type * px_, boost::detail::shared_count && pn_ ) BOOST_SP_NOEXCEPT : px( px_ ), pn( std::move( pn_ ) )
                                                                                                                                                                        ~~~~~^
/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/smart_ptr/shared_ptr.hpp:927:27: error: no member named 'move' in namespace 'std'; did you mean 'modf'?
    return shared_ptr<T>( std::move(r), p );
                          ^~~~~~~~~
                          modf
/usr/include/math.h:407:15: note: 'modf' declared here
extern double modf(double, double *);
              ^
In file included from /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp:646:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan_fit.hpp:12:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/posix_time/posix_time_types.hpp:10:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/time_clock.hpp:17:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/shared_ptr.hpp:17:
/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/smart_ptr/shared_ptr.hpp:937:27: error: no member named 'move' in namespace 'std'; did you mean 'modf'?
    return shared_ptr<T>( std::move(r), p );

                          modf
/usr/include/math.h:407:15: note: 'modf' declared here
extern double modf(double, double *);
              ^

So what is going on here? I have the latest version of pystan, I have updated conda… what is going wrong?

[ahartikainen]

Hi, can you see https://github.com/stan-dev/pystan/blob/develop/doc/installation_beginner.rst (C++ section)

I think you either are missing libc++ (or some headers) or you compiler is too old.

@pbadel

I have installed conda in my personal directories: python2 in /Users/peter/miniconda2 and python3 in /Users/peter/miniconda3 . My paths are set up such that “python” gives me /Users/peter/miniconda2/bin/python and “python3” gives me /Users/peter/miniconda3/bin/python . I get the same error below whether I use python2 or python3.

Small comment about this setup:

I don’t think that is recommended and probably will even fail. There is more to conda than just adding python the the path.

You really only need one conda installed and then you should create conda environments which could be python 2 or python 3.

conda create -n py36 python=3.6
conda create -n py27 python=3.6

and then when you need python 3

conda activate py36

or

source activate py36

and then use python which is python 3.6

To deactivate you can do

conda deactivate

[pbadel]

Thanks for your feedback about my Conda setup. You are probably correct, but I don’t understand activate and conda environments, and I’d rather not fiddle with my Python setup when it works for every other 3rd party Python module I use. PyStan seems to be a bit of a special case among Python modules, because it calls the GCC compiler. (Perhaps cython is another exception, but that works for me…)

So I am abandoning PyStan and using CmdStan instead.

@ahartikainen

There is more to conda than just adding python the the path.

I’m straying off topic, but rightly or wrongly, I have seen that advice elsewhere, e.g. How do I set Anaconda's python as my default python command? - Stack Overflow 14.04 - How to set anaconda as a default python? - Ask Ubuntu

[ahartikainen]

@pbadel

You are probably correct, but I don’t understand activate and conda environments

After initial learning bump, conda is quite pleasent to use. (basically you have independent directories that contain all python stuff, and calling source activate envname /(or newer conda installations give you option to call conda activate envname) will make the envfolder the main (updates $PATH for bin and python folders).

To see all the environments, you can use conda env list.

@pbadel

using CmdStan

We also have python interface to CmdStan called CmdStanPy https://cmdstanpy.readthedocs.io/en/latest/

And ArviZ supports both CmdStan/CmdStanPy too, so you can do visualizations and diagnostics with it, if you want.

https://arviz-devs.github.io/arviz/index.html

But I don’t think your PyStan is failing due your Python setup.

The problem is with your compiler and this error should also manifest itself with CmdStan too.

So, I think you should try to fix the header issue with the macOS compiler.

See the following resources

Dealing with Mojave

For the first time, getting rstan to work on Windows seems to be easier than getting it to work on the Mojave version Mac OS X. Mojave seems to affect different people in different ways. Here are the things I would try if rstan is not working: Make sure you have opened Xcode once via Spotlight to accept its license If that doesn’t work but you have a /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14. pkg, install that so it puts C++ headers back in /usr/include I…

Stan with Mojave MacOS

Hi everyone, I recently updated my MacOS software to Mojave and since I have not been able to use Stan with R. When I try loading the ‘rstan’ package, I get the following error: library(rstan) Loading required package: StanHeaders Error in value[3L] : Package ‘StanHeaders’ version 2.17.2 cannot be unloaded: Error in unloadNamespace(package) : namespace ‘StanHeaders’ is imported by ‘rstan’ so cannot be unloaded I have tried to look at re-installing RStan, but when I execute pkgbuild::ha…

Compilation error in Pystan (macOS Mojave)

Hi, First, use pip version pip install pystan and try compiling again. There seems to be similar problems with mojave Run this (in pandas issue) and try again: /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg If that doesn’t work, then remove xcode and commandline tools and do the fresh install. Run all these tests in fresh conda-env. Install PyStan from source to a new conda env. conda create -n stan_env python=3.7 numpy cython conda activate sta…

Make sure you have opened Xcode once via Spotlight to accept its license

/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

install that so it puts C++ headers back in /usr/include

xcode-select --install

[pbadel]

@ahartikainen

But I don’t think your PyStan is failing due your Python setup.

The problem is with your compiler and this error should also manifest itself with CmdStan too.

CmdStan is working fine for me.

It looks like most CmdStan compiling was done with clang++, and clang++ --version gives:

Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

As for PyStan, I suspect there was something strange with library paths inside Python (or inside PyStan). But since I have a working solution, I won’t be looking into it further at the moment.