When building cmdstan2.18 I received this message:
gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-5.4.0/release/threadapi-pthread/threading-multi/list.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
from ./boost/python/list.hpp:8,
from libs/python/src/list.cpp:5:
./boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
compilation terminated.
Any suggestions how to build cmdstan?
This morning I cloned cmdstan with
git clone https://github.com/stan-dev/cmdstan.git --recursive
changed make/local file to
STAN_MPI=TRUE
CC=mpicxx
and run
make build
Ubuntu 16.04.5 LTS
gcc version 5.4.0
I am using openmpi.
If you cloned it today, you need to change CC= to CXX=. It’s updated in the CmdStan doc that lives within the repo (you’ll need LaTeX to compile it). There was an overhaul to the makefiles.
ar -rs stan/lib/stan_math/lib/sundials_3.1.0/lib/libsundials_idas.a …
then boost is built but for some reason it uses:
gcc.compile.c++
and it breaks on:
gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-5.4.0/release/threadapi-pthread/threading-multi/list.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
from ./boost/python/list.hpp:8,
from libs/python/src/list.cpp:5:
./boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
Just to double check, you started from a clean CmdStan? If not, would you mind typing this (I’m assuming you’re in CmdStan’s home folder and you’ve used git to clone everything):
cd stan/lib/stan_math
git clean -d -x -f
cd …/…/…/
make stan/lib/stan_math/lib/boost_1.66.0/stage/lib/libboost_serialization.dylib
Could you copy what happens (all the output) from the last line there? I want to see if there’s a difference in how mine is set up.
gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-5.4.0/release/threadapi-pthread/threadin g-multi/list.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
from ./boost/python/list.hpp:8,
from libs/python/src/list.cpp:5:
./boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
compilation terminated.
For example… this is the sort of output I’m looking for:
$ make stan/lib/stan_math/lib/boost_1.66.0/stage/lib/libboost_serialization.dylib
echo "# In case of a compiler mismatch used by mpicxx and" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "# the compiler used for Stan, consider configuring" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "# the boost toolset here" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "# Moreover, should your mpicxx command live in a" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "# in a non-standard directory, then consider to tell" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "# boost mpi using this syntax:" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "#using mpi : /path/to/mpicxx ;" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo "using mpi ;" >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
cd stan/lib/stan_math/lib/boost_1.66.0; ./bootstrap.sh
Building Boost.Build engine with toolset darwin... tools/build/src/engine/bin.macosxx86_64/b2
Detecting Python version... 2.7
Detecting Python root... /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7
Unicode/ICU support for Boost.Regex?... not found.
Generating Boost.Build configuration in project-config.jam...
Bootstrapping is done. To build, run:
./b2
To adjust configuration, edit 'project-config.jam'.
Further information:
- Command line help:
./b2 --help
- Getting started guide:
[http://www.boost.org/more/getting_started/unix-variants.html](http://www.boost.org/more/getting_started/unix-variants.html)
- Boost.Build documentation:
[http://www.boost.org/build/doc/html/index.html](http://www.boost.org/build/doc/html/index.html)
cd stan/lib/stan_math/lib/boost_1.66.0; ./b2 --user-config=user-config.jam --layout=system --with-mpi --with-serialization -j1 variant=release link=shared threading=multi runtime-link=shared hardcode-dll-paths=true dll-path="/Users/daniel/stan-dev/cmdstan/stan/lib/stan_math/lib/boost_1.66.0/stage/lib"
Performing configuration checks
- 32-bit : no
- 64-bit : yes
- arm : no
- mips1 : no
- power : no
- sparc : no
- x86 : yes
Building the Boost C++ Libraries.
Component configuration:
- atomic : not building
- chrono : not building
- date_time : not building
- exception : not building
- graph : not building
- mpi : building
- python : not building
- regex : not building
- serialization : building
- system : not building
- thread : not building
- timer : not building
...patience...
echo “# In case of a compiler mismatch used by mpicxx and” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “# the compiler used for Stan, consider configuring” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “# the boost toolset here” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “# Moreover, should your mpicxx command live in a” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “# in a non-standard directory, then consider to tell” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “# boost mpi using this syntax:” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “#using mpi : /path/to/mpicxx ;” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
echo “using mpi ;” >> stan/lib/stan_math/lib/boost_1.66.0/user-config.jam
cd stan/lib/stan_math/lib/boost_1.66.0; ./bootstrap.sh
Building Boost.Build engine with toolset gcc… tools/build/src/engine/bin.linuxx86_64/b2
Detecting Python version… 2.7
Detecting Python root… /usr
Unicode/ICU support for Boost.Regex?.. not found.
Generating Boost.Build configuration in project-config.jam…
Bootstrapping is done. To build, run:
./b2
To adjust configuration, edit ‘project-config.jam’.
Further information:
- 32-bit : no
- 64-bit : yes
- arm : no
- mips1 : no
- power : no
- sparc : no
- x86 : yes
Building the Boost C++ Libraries.
Component configuration:
- atomic : not building
- chrono : not building
- date_time : not building
- exception : not building
- graph : not building
- mpi : building
- python : not building
- regex : not building
- serialization : building
- system : not building
- thread : not building
- timer : not building
…failed gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-5.4.0/release/threadapi-pthread/threading-multi/list.o…
gcc.compile.c++ bin.v2/libs/python/build/gcc-gnu-5.4.0/release/threadapi-pthread/threading-multi/long.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
from ./boost/python/long.hpp:8,
from libs/python/src/long.cpp:5:
./boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
compilation terminated.
I also faced this problem, although my machine had python-devel.
At that time, I used miniconda3-4.3.11(python 3.6.1) with pyenv, so my python commands ran python3.
After I changed python commands so that it ran python2, this error solved.
Maybe it makes sense to mention in the documentation what development tools have to be installed? While g++ is obvious python may be not. Looks like python is required to compile boost and since it comes packaged with stan one has no clue that python development package is needed. Also, there is a practice to package python and mpi separately from development tools.