Hey all
I have encountered problems with installing cmdstan on a linux server. The server has mutiple toolchains which may be activated. I am using
CmdStan: 2.34.0
gcc: 9.4.0
gnu make: 3.82
running make build -j4 from the terminal yields the output Cmdstan v2.34.0 built which indicates the build was successful. However when testing the build with make examples/bernoulli/bernoulli the linking gives the output
src/cmdstan/main.o: In function `cmdstan::command(int, char const**)':
main.cpp:(.text+0x1276e): undefined reference to `tbb::interface7::internal::task_arena_base::internal_initialize()'
main.cpp:(.text+0x13096): undefined reference to `tbb::interface9::global_control::internal_create()'
src/cmdstan/main.o: In function `tbb::interface7::task_arena::~task_arena()':
main.cpp:(.text._ZN3tbb10interface710task_arenaD2Ev[_ZN3tbb10interface710task_arenaD5Ev]+0x15): undefined reference to `tbb::interface7::internal::task_arena_base::internal_terminate()'
src/cmdstan/main.o: In function `tbb::interface9::global_control::~global_control()':
main.cpp:(.text._ZN3tbb10interface914global_controlD2Ev[_ZN3tbb10interface914global_controlD5Ev]+0x5): undefined reference to `tbb::interface9::global_control::internal_destroy()'
collect2: error: ld returned 1 exit status
make: *** [examples/bernoulli/bernoulli] Error 1
Any advice on why this may occur and how to fix it?
Does the server provide a version of intel’s TBB library you’re using? i.e. have you edited make/local or are environment variables like TBB or TBB_LIB set?
Thanks for the response Brian. I am still a beginner in the use of linux, and so I am not familiar with tbb at all. I thus do not know how/where to check for this on the server, and I have not edited the make/local file. I just followed the instructions on the cmdstan reference manual
You can use the command “printenv” to inspect the server’s environment. Look for any variables with “TBB” or “LD” in the name
The command printenv | tr ' ' '\n' | grep -i 'LD' displays the imported gcc module LD_LIBRARY_PATH=/apps/lib/boost/1.80.0-gcc-9.4:/apps/gcc/9.4.0/lib:/apps/gcc/9.4.0/lib64, and printenv | tr ' ' '\n' | grep -i 'TBB' displays nothing. The default gcc module on the server is 4.8.3 thus I have to import a later version using module load <compiler>
One more environment variable to check: does LIBRARY_PATH (without LD_) have anything?
Can you post the full output from make examples/bernoulli/bernoulli (not just the error?)
LIBRARY_PATH also has nothing
The full output is below
make examples/bernoulli/bernoulli
--- Translating Stan model to C++ code ---
bin/stanc --o=examples/bernoulli/bernoulli.hpp examples/bernoulli/bernoulli.stan
--- Compiling C++ code ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.81.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DBOOST_DISABLE_ASSERTS -c -Wno-ignored-attributes -x c++ -o examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp
--- Linking model ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.81.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DBOOST_DISABLE_ASSERTS -Wl,-L,"/home/22796002/cmdstan-2.34.0/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/home/22796002/cmdstan-2.34.0/stan/lib/stan_math/lib/tbb" examples/bernoulli/bernoulli.o src/cmdstan/main.o -ltbb stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a stan/lib/stan_math/lib/tbb/libtbb.so.2 -o examples/bernoulli/bernoulli
src/cmdstan/main.o: In function `cmdstan::command(int, char const**)':
main.cpp:(.text+0x1276e): undefined reference to `tbb::interface7::internal::task_arena_base::internal_initialize()'
main.cpp:(.text+0x13096): undefined reference to `tbb::interface9::global_control::internal_create()'
src/cmdstan/main.o: In function `tbb::interface7::task_arena::~task_arena()':
main.cpp:(.text._ZN3tbb10interface710task_arenaD2Ev[_ZN3tbb10interface710task_arenaD5Ev]+0x15): undefined reference to `tbb::interface7::internal::task_arena_base::internal_terminate()'
src/cmdstan/main.o: In function `tbb::interface9::global_control::~global_control()':
main.cpp:(.text._ZN3tbb10interface914global_controlD2Ev[_ZN3tbb10interface914global_controlD5Ev]+0x5): undefined reference to `tbb::interface9::global_control::internal_destroy()'
collect2: error: ld returned 1 exit status
make: *** [examples/bernoulli/bernoulli] Error 1
rm examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp
I have managed to get the installation working. My main aim was to get cmdstanpy installed, but for this I needed a working cmdstan installation. The easiest and most reliable method to getting this installed is through conda, and the conda-forge channel
- Create a new env and install cmdstan
conda -n create stan -c conda-forge cmdstan=2.34.0
- The above installation may still require that cmdstan be built. So I recommend navigating to the
<cmdstan home> directory which should be conda/envs/stan/bin/cmdstan. We now remove the built stan files and rebuild it
conda activate stan
cd conda/envs/stan/bin/cmdstan
make clean-all
make build
- Since cmdstan was installed through conda-forge, it seems like all other packages should be installed through conda-forge as well, otherwise it results in errors regarding evironment conflicts. We thus install python and cmdstanpy through the conda-forge channel
conda activate stan
conda install -c conda-forge python cmdstanpy
By installing cmdstan first through conda-forge the cmdstanpath when installing cmdstanpy is automatically updated as well, thus no need to run any lines for manually setting the cmdstan_path.
Following these steps resulted in a working cmdstan and cmdstanpy installation without having to set any environmental variables (or make/local files) manually.
I previously tried to install cmdstan by cloning from github, but the complex architecture of the server (somehow) resulted in conflicts with the TBB libraries from cmdstan. By installing cmdstan from conda-forge, it downloads a compatible gcc compiler and also edits the environmental variables automatically to only use the gcc compiler downloaded as well as automatically generating a make/local file compatible with the server. This installation is valid across windows, linux and linux servers (tried and tested on everything).
Maybe the documentation should be updated to add these instructions for installing cmdstan/cmdstanpy across platforms??
Hopefully someone finds this useful in the future
The conda-forge option is described in both the cmdstan and cmdstanpy docs. I’m glad it worked better for you!
Note there should not be any need to re-build the conda-provided cmdstan (so you can skip the make clean-all step you describe)
1 Like