Pystan causes crash (gcc compiler?), conda on Mac OS X

Mac OS X 10.12.5, miniconda, python 3.6.8, pystan 2.19.0.0. I don’t know what Compiler/Toolkit I am using.

I have installed conda in my personal directories: python2 in /Users/peter/miniconda2 and python3 in /Users/peter/miniconda3 . My paths are set up such that “python” gives me /Users/peter/miniconda2/bin/python and “python3” gives me /Users/peter/miniconda3/bin/python . I get the same error below whether I use python2 or python3.

I am trying to run the “Getting started, Example 1” example at https://pystan.readthedocs.io/en/latest/getting_started.html (except I have added “verbose=True” to the line which gives the error), at the line

sm = pystan.StanModel(model_code=schools_code, verbose=True)

I get a long output and eventually the “too many errors” error. So I will just give the log to the first two errors.

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_19a09b474d1901f191444eaf8a6b8ce2 NOW.
INFO:pystan:OS: darwin, Python: 3.6.8 |Anaconda, Inc.| (default, Dec 29 2018, 19:04:46) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)], Cython 0.28.5
Compiling /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.pyx because it changed.
[1/1] Cythonizing /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.pyx
building 'stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416' extension
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T
creating /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/peter/miniconda3/include -arch x86_64 -I/Users/peter/miniconda3/include -arch x86_64 -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3 -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/src -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/eigen_3.3.3 -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0 -I/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include -I/Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include -I/Users/peter/miniconda3/include/python3.6m -c /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp -o /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.o -O2 -ftemplate-depth-256 -Wno-unused-function -Wno-uninitialized -std=c++1y
In file included from /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp:642:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:
/Users/peter/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
 ^
In file included from /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp:646:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan_fit.hpp:12:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/posix_time/posix_time_types.hpp:10:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/time_clock.hpp:17:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/shared_ptr.hpp:17:
/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/smart_ptr/shared_ptr.hpp:370:174: error: no member named 'move' in namespace 'std'
    BOOST_CONSTEXPR shared_ptr( boost::detail::sp_internal_constructor_tag, element_type * px_, boost::detail::shared_count && pn_ ) BOOST_SP_NOEXCEPT : px( px_ ), pn( std::move( pn_ ) )
                                                                                                                                                                        ~~~~~^
/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/smart_ptr/shared_ptr.hpp:927:27: error: no member named 'move' in namespace 'std'; did you mean 'modf'?
    return shared_ptr<T>( std::move(r), p );
                          ^~~~~~~~~
                          modf
/usr/include/math.h:407:15: note: 'modf' declared here
extern double modf(double, double *);
              ^
In file included from /var/folders/6l/mw9lfs_51h74dfvg458rrn1c0000gp/T/tmpw103deo3/stanfit4anon_model_19a09b474d1901f191444eaf8a6b8ce2_6187777314581226416.cpp:646:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan_fit.hpp:12:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/posix_time/posix_time_types.hpp:10:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/date_time/time_clock.hpp:17:
In file included from /Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/shared_ptr.hpp:17:
/Users/peter/miniconda3/lib/python3.6/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/smart_ptr/shared_ptr.hpp:937:27: error: no member named 'move' in namespace 'std'; did you mean 'modf'?
    return shared_ptr<T>( std::move(r), p );

                          modf
/usr/include/math.h:407:15: note: 'modf' declared here
extern double modf(double, double *);
              ^

So what is going on here? I have the latest version of pystan, I have updated conda… what is going wrong?

Hi, can you see https://github.com/stan-dev/pystan/blob/develop/doc/installation_beginner.rst (C++ section)

I think you either are missing libc++ (or some headers) or you compiler is too old.

Small comment about this setup:

I don’t think that is recommended and probably will even fail. There is more to conda than just adding python the the path.

You really only need one conda installed and then you should create conda environments which could be python 2 or python 3.

conda create -n py36 python=3.6
conda create -n py27 python=3.6

and then when you need python 3

conda activate py36

or

source activate py36

and then use python which is python 3.6

To deactivate you can do

conda deactivate

Thanks for your feedback about my Conda setup. You are probably correct, but I don’t understand activate and conda environments, and I’d rather not fiddle with my Python setup when it works for every other 3rd party Python module I use. PyStan seems to be a bit of a special case among Python modules, because it calls the GCC compiler. (Perhaps cython is another exception, but that works for me…)

So I am abandoning PyStan and using CmdStan instead.

I’m straying off topic, but rightly or wrongly, I have seen that advice elsewhere, e.g. https://stackoverflow.com/questions/36075166/how-do-i-set-anacondas-python-as-my-default-python-command https://askubuntu.com/questions/886983/how-to-set-anaconda-as-a-default-python

After initial learning bump, conda is quite pleasent to use. (basically you have independent directories that contain all python stuff, and calling source activate envname /(or newer conda installations give you option to call conda activate envname) will make the envfolder the main (updates $PATH for bin and python folders).

To see all the environments, you can use conda env list.

We also have python interface to CmdStan called CmdStanPy cmdstanpy – Python interface to CmdStan — CmdStanPy 0.9.66 documentation

And ArviZ supports both CmdStan/CmdStanPy too, so you can do visualizations and diagnostics with it, if you want.

https://arviz-devs.github.io/arviz/index.html

But I don’t think your PyStan is failing due your Python setup.

The problem is with your compiler and this error should also manifest itself with CmdStan too.

So, I think you should try to fix the header issue with the macOS compiler.

See the following resources

Make sure you have opened Xcode once via Spotlight to accept its license
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

install that so it puts C++ headers back in /usr/include

xcode-select --install

CmdStan is working fine for me.

It looks like most CmdStan compiling was done with clang++, and clang++ --version gives:

Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

As for PyStan, I suspect there was something strange with library paths inside Python (or inside PyStan). But since I have a working solution, I won’t be looking into it further at the moment.