cmdStan Intel TBB error when compiling model

mjcarter · January 27, 2025, 1:10pm

I run into an Intel TBB linking error when trying to compile the Bernoulli example model using a fresh installation of cmdStan.

Operating System: CentOS Linux 7.9.2009
GCC Compiler: 11.2.0
cmdStan: v2.36.0

I configure my environment on the HPC server using:

module load compilers/gcc/11.2.0

and then download / build cmdStan with:

wget https://github.com/stan-dev/cmdstan/releases/download/v2.36.0/cmdstan-2.36.0.tar.gz
tar -xzf cmdstan-2.36.0.tar.gz
cd cmdstan-2.36.0
make build

which builds successfully without error

--- CmdStan v2.36.0 built ---

However, when trying to compile the Bernoulli model with:

make examples/bernoulli/bernoulli

I see the following error:

/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++ -std=c++17 -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/include    -DBOOST_DISABLE_ASSERTS          -c -MT stan/src/stan/model/model_header.hpp.gch/model_header_11_2.hpp.gch -M -E -MG -MP -MF stan/src/stan/model/model_header.hpp.gch/model_header_11_2.d stan/src/stan/model/model_header.hpp
/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++ -std=c++17 -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/include    -DBOOST_DISABLE_ASSERTS          -c -MT src/cmdstan/main.o -MM -E -MG -MP -MF src/cmdstan/main.d src/cmdstan/main.cpp

--- Translating Stan model to C++ code ---
bin/stanc  --o=examples/bernoulli/bernoulli.hpp examples/bernoulli/bernoulli.stan

--- Compiling C++ code ---
/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++ -std=c++17 -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/include    -DBOOST_DISABLE_ASSERTS          -c -Wno-ignored-attributes   -x c++ -o examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp

--- Linking model ---
/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++ -std=c++17 -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/include    -DBOOST_DISABLE_ASSERTS         -L/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib64       -Wl,-L,"/mnt/data1/users/mcarter/cmdstan/stan/lib/stan_math/lib/tbb"   -Wl,-rpath,"/mnt/data1/users/mcarter/cmdstan/stan/lib/stan_math/lib/tbb"      examples/bernoulli/bernoulli.o src/cmdstan/main.o       -ltbb   stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a  stan/lib/stan_math/lib/tbb/libtbb.so.2 -o examples/bernoulli/bernoulli
src/cmdstan/main.o: In function `cmdstan::command(int, char const**)':
main.cpp:(.text+0x2db7f): undefined reference to `tbb::interface7::internal::task_arena_base::internal_initialize()'
main.cpp:(.text+0x2f8fc): undefined reference to `tbb::interface9::global_control::internal_create()'
src/cmdstan/main.o: In function `tbb::interface7::task_arena::~task_arena()':
main.cpp:(.text._ZN3tbb10interface710task_arenaD2Ev[_ZN3tbb10interface710task_arenaD5Ev]+0x15): undefined reference to `tbb::interface7::internal::task_arena_base::internal_terminate()'
src/cmdstan/main.o: In function `tbb::interface9::global_control::~global_control()':
main.cpp:(.text._ZN3tbb10interface914global_controlD2Ev[_ZN3tbb10interface914global_controlD5Ev]+0x5): undefined reference to `tbb::interface9::global_control::internal_destroy()'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::invalid_argument@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::length_error::length_error(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::range_error::range_error(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_allocate_exception@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::bad_alloc::~bad_alloc()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::string::_Rep::_M_destroy(std::allocator<char> const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::bad_alloc::~bad_alloc()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_throw@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::string::_Rep::_M_dispose(std::allocator<char> const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `operator delete(void*)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::runtime_error::runtime_error(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::runtime_error::~runtime_error()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::length_error::~length_error()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::range_error::~range_error()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `operator new(unsigned long)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `vtable for __cxxabiv1::__class_type_info@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_end_catch@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_rethrow@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::out_of_range@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::string::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::invalid_argument::~invalid_argument()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::length_error@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_call_unexpected@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__gxx_personality_v0@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::out_of_range::out_of_range(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_free_exception@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `operator delete[](void*)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::bad_alloc@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::range_error@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::exception::~exception()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::exception@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::runtime_error@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::out_of_range::~out_of_range()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `vtable for __cxxabiv1::__si_class_type_info@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_begin_catch@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `vtable for __cxxabiv1::__vmi_class_type_info@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `operator new[](unsigned long)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_pure_virtual@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::invalid_argument::invalid_argument(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `vtable for std::bad_alloc@GLIBCXX_3.4'
collect2: error: ld returned 1 exit status
make: *** [examples/bernoulli/bernoulli] Error 1
rm examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp

I’m a bit confused why these errors don’t appear whilst compiling cmdStan. Any suggestions?

Bob_Carpenter · January 27, 2025, 4:18pm

This is one for @stevebronder.

The only hints on our documentation page

https://mc-stan.org/docs/cmdstan-guide/installation.html

are for Windows.

WardBrian · January 27, 2025, 9:17pm

/lib/../lib64/libtbb.so

This implies TBB is being linked from the HPC system, not from CmdStan’s copy. This is actually probably a good thing, if you can get it working. The errors look like what happens when you’re trying to link a newer version of TBB than we expect, but we can ask cmdstan to use a newer version

On my cluster, this usually looks like

module load compilers/gcc/11.2.0
module load intel/oneTBB # consult your HPC's specific name

and then in cmdstan/make/local, set

TBB_CXX_TYPE=gcc
TBB_INTERFACE_NEW=true
TBB_INC=$(TBBROOT)/include/
TBB_LIB=$(TBBROOT)/lib/

(for me, my module system sets TBBROOT, ymmv)

mjcarter · January 28, 2025, 11:04am

Intel TBB is not installed as a package on the HPC system that I am using. A number of packages that ship with Intel TBB are available, but I have not loaded any of these (see below), so they shouldn’t change things in my environment.

Currently Loaded Modulefiles:
  1) null                   2) compilers/gcc/11.2.0

Is it possible to force Stan to compile against the packaged version of Intel TBB?

WardBrian · January 28, 2025, 2:41pm

Can you try compiling with CXXFLAGS=-Wl,-v make examples/bernoulli/bernoulli?

It may also be worth sharing the output of make print-compiler-flags

We need to understand why the system is picking up a different TBB before it’s possible to make it use another

mjcarter · January 28, 2025, 3:26pm

Hi Brian,

Below is the output with CXXFLAGS=-Wl,-v make ...

[mcarter@viz02[barkla] cmdstan-2.36.0]$ CXXFLAGS=-Wl,-v make examples/bernoulli/bernoulli

--- Translating Stan model to C++ code ---
bin/stanc  --o=examples/bernoulli/bernoulli.hpp examples/bernoulli/bernoulli.stan

--- Compiling C++ code ---
/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++ -Wl,-v -std=c++17 -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/include    -DBOOST_DISABLE_ASSERTS   -DTBB_INTERFACE_NEW  -DTBB_INTERFACE_NEW     -c -Wno-ignored-attributes   -x c++ -o examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp

--- Linking model ---
/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++ -Wl,-v -std=c++17 -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/include    -DBOOST_DISABLE_ASSERTS   -DTBB_INTERFACE_NEW  -DTBB_INTERFACE_NEW    -L/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib64       -Wl,-L,"/mnt/data1/users/mcarter/cmdstan-2.36.0/stan/lib/stan_math/lib/tbb"   -Wl,-rpath,"/mnt/data1/users/mcarter/cmdstan-2.36.0/stan/lib/stan_math/lib/tbb"      examples/bernoulli/bernoulli.o src/cmdstan/main.o       -ltbb   stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a  stan/lib/stan_math/lib/tbb/libtbb.so.2 -o examples/bernoulli/bernoulli
collect2 version 11.2.0
/usr/bin/ld -plugin /opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/libexec/gcc/x86_64-pc-linux-gnu/11.2.0/liblto_plugin.so -plugin-opt=/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/libexec/gcc/x86_64-pc-linux-gnu/11.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccekrBTP.res -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o examples/bernoulli/bernoulli /lib/../lib64/crt1.o /lib/../lib64/crti.o /opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/crtbegin.o -L/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib64 -L/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0 -L/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../.. -v -L /mnt/data1/users/mcarter/cmdstan-2.36.0/stan/lib/stan_math/lib/tbb -rpath /mnt/data1/users/mcarter/cmdstan-2.36.0/stan/lib/stan_math/lib/tbb examples/bernoulli/bernoulli.o src/cmdstan/main.o -ltbb stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a stan/lib/stan_math/lib/tbb/libtbb.so.2 -lstdc++ -lm -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc /opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/lib/gcc/x86_64-pc-linux-gnu/11.2.0/crtend.o /lib/../lib64/crtn.o
GNU ld version 2.27-44.base.el7_9.1
src/cmdstan/main.o: In function `cmdstan::command(int, char const**)':
main.cpp:(.text+0x2d92f): undefined reference to `tbb::interface7::internal::task_arena_base::internal_initialize()'
main.cpp:(.text+0x2f6ac): undefined reference to `tbb::interface9::global_control::internal_create()'
src/cmdstan/main.o: In function `tbb::interface7::task_arena::~task_arena()':
main.cpp:(.text._ZN3tbb10interface710task_arenaD2Ev[_ZN3tbb10interface710task_arenaD5Ev]+0x15): undefined reference to `tbb::interface7::internal::task_arena_base::internal_terminate()'
src/cmdstan/main.o: In function `tbb::interface9::global_control::~global_control()':
main.cpp:(.text._ZN3tbb10interface914global_controlD2Ev[_ZN3tbb10interface914global_controlD5Ev]+0x5): undefined reference to `tbb::interface9::global_control::internal_destroy()'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::invalid_argument@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::length_error::length_error(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::range_error::range_error(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_allocate_exception@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::bad_alloc::~bad_alloc()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::string::_Rep::_M_destroy(std::allocator<char> const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::bad_alloc::~bad_alloc()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_throw@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::string::_Rep::_M_dispose(std::allocator<char> const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `operator delete(void*)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::runtime_error::runtime_error(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::runtime_error::~runtime_error()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::length_error::~length_error()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::range_error::~range_error()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `operator new(unsigned long)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `vtable for __cxxabiv1::__class_type_info@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_end_catch@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_rethrow@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::out_of_range@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::string::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::invalid_argument::~invalid_argument()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::length_error@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_call_unexpected@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__gxx_personality_v0@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::out_of_range::out_of_range(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_free_exception@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `operator delete[](void*)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::bad_alloc@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::range_error@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::exception::~exception()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::exception@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `typeinfo for std::runtime_error@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::out_of_range::~out_of_range()@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `vtable for __cxxabiv1::__si_class_type_info@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_begin_catch@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `vtable for __cxxabiv1::__vmi_class_type_info@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `operator new[](unsigned long)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `__cxa_pure_virtual@CXXABI_1.3'
/lib/../lib64/libtbb.so: undefined reference to `std::invalid_argument::invalid_argument(std::string const&)@GLIBCXX_3.4'
/lib/../lib64/libtbb.so: undefined reference to `vtable for std::bad_alloc@GLIBCXX_3.4'
collect2: error: ld returned 1 exit status
make: *** [examples/bernoulli/bernoulli] Error 1
rm examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp

And for make print-compiler-flags

 Current configuration:
  - OS (Operating System):       Linux
  - CXX (Compiler):              /opt/gridware/depots/e2b91392/el7/pkg/compilers/gcc/11.2.0/bin/g++
  - CXX_TYPE                     gcc
  - Compiler version:            11.2
  - O (Optimization Level):      3
  Library configuration:
  - EIGEN                        stan/lib/stan_math/lib/eigen_3.4.0
  - BOOST                        stan/lib/stan_math/lib/boost_1.84.0
  - SUNDIALS                     stan/lib/stan_math/lib/sundials_6.1.1
  - TBB                          stan/lib/stan_math/lib/tbb_2020.3
  - GTEST                        stan/lib/stan_math/lib/benchmark_1.5.1/googletest/googletest
  - STAN_THREADS                
  - STAN_OPENCL                 
  - STAN_MPI                    
  Compiler flags (each can be overriden separately):
  - CXXFLAGS_LANG                -std=c++17
  - CXXFLAGS_WARNINGS            -Wno-sign-compare -Wno-ignored-attributes -Wno-class-memaccess
  - CXXFLAGS_OPTIM              
  - CXXFLAGS_FLTO               
  - CXXFLAGS_BOOST              
  - CXXFLAGS_EIGEN              
  - CXXFLAGS_OS                  -pthread -D_REENTRANT
  - CXXFLAGS_GTEST              
  - CXXFLAGS_THREADS            
  - CXXFLAGS_OPENCL             
  - CXXFLAGS_TBB                 -I stan/lib/stan_math/lib/tbb_2020.3/include
  - CXXFLAGS_OPTIM_TBB          
  - CXXFLAGS_MPI                
  - CFLAGS_SUNDIALS             
  - CXXFLAGS_SUNDIALS            -pipe
  - CXXFLAGS_OPTIM_SUNDIALS     
  LDLIBS:
  - LDLIBS_LANG                 
  - LDLIBS_WARNINGS             
  - LDLIBS_EIGEN                
  - LDLIBS_BOOST                
  - LDLIBS_SUNDIALS             
  - LDLIBS_OS                   
  - LDLIBS_GTEST                
  - LDLIBS_OPENCL               
  - LDLIBS_TBB                   -ltbb
  - LDLIBS_MPI                  
  LDFLAGS:
  - LDFLAGS_LANG                
  - LDFLAGS_WARNINGS            
  - LDFLAGS_EIGEN               
  - LDFLAGS_BOOST               
  - LDFLAGS_SUNDIALS            
  - LDFLAGS_OS                  
  - LDFLAGS_GTEST               
  - LDFLAGS_OPENCL              
  - LDFLAGS_TBB                  -Wl,-L,/mnt/data1/users/mcarter/cmdstan-2.36.0/stan/lib/stan_math/lib/tbb -Wl,-rpath,/mnt/data1/users/mcarter/cmdstan-2.36.0/stan/lib/stan_math/lib/tbb
  - LDFLAGS_MPI

WardBrian · January 28, 2025, 3:32pm

Ok, so it looks like LD is being passed -L/lib/../lib64 , which contains libtbb.so. I’m guessing this comes from either LDLIBS or LDFLAGS in your environment variables.

I believe all you need to do is make sure that the TBB from CmdStan is listed earlier in the list, so you might be able to get away with something like this in make/local

CUSTOM_LDFLAGS = $(LDFLAGS_TBB) + $(LDFLAGS)
override LDFLAGS = $(CUSTOM_LDFLAGS)

stevebronder · January 28, 2025, 7:39pm

As Brian said, from the error message from the compiler it looks like tbb is still loaded on your instance even if you are not directly loading it. Can you run env and look for TBBROOT? If that exists then you can use what brian posted above.

TBB_CXX_TYPE=gcc
TBB_INTERFACE_NEW=true
TBB_INC=$(TBBROOT)/include/
TBB_LIB=$(TBBROOT)/lib/

If that does not work can you run ls ../lib64 and look for libtbb.so.2 (or just libtbb.so)? If that is there then you will also have to change the above to something like the below, where the include folder is usually in either /include/ or /include/tbb/{VERSION_NUMBER} where {VERSION_NUMBER} is something like 2021.01.12 etc.

TBB_CXX_TYPE=gcc
TBB_INTERFACE_NEW=true
TBB_INC=/include/
TBB_LIB=/lib64/

If neither of those two work then you should contact your hpc admin as they will know where those are in your system and can give you the correct directories. For us it’s kind of whackamole without knowing what your systems directory structure looks like.

Topic		Replies	Views
Unable to compile model using cmdstan on linux CmdStan	2	454	July 4, 2023
Cmdstan 2.24.1 Compile Error CmdStan	3	673	September 21, 2020
Cmdstan 2.21 windows can't make bernoulli example CmdStan	14	1670	November 4, 2019
Unable to compile `Bernoulli` example on macOS Monterey 12.6.1 CmdStan example-models	2	461	November 15, 2022
Issue compiling bernoulli example model in cmdstan 2.17 CmdStan	14	1721	May 9, 2018

cmdStan Intel TBB error when compiling model

Related topics