Unable to build cmdstan 2.30.1 on Debian bullseye - "bits/c++config.h" - tbbmalloc

Interfaces CmdStan
1

System information:

$> uname -a
Linux 5.13.4-x86_64-linode146 #1 SMP Wed Jul 21 14:45:45 EDT 2021 x86_64 GNU/Linux
$> cat /etc/debian_version
11.5

Sadly CmdStan does not seem to compile on Debian bullseye; Intel Threading Building Blocks seems to be the culprit:

make[1]: Entering directory '/tmp/cmdstan-2.30.1/stan/lib/stan_math/lib/tbb'
g++ -c -MMD -O2 -g -DDO_ITT_NOTIFY -DUSE_PTHREAD -pthread -m64 -mrtm   -Wno-unknown-warning-option -Wno-deprecated-copy -Wno-missing-attributes -Wno-class-memaccess -Wno-sized-deallocation   -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fno-rtti -fno-exceptions -D__TBBMALLOC_BUILD=1 -Wno-parentheses -Wno-sized-deallocation -fPIC -flifetime-dse=1 -I../tbb_2020.3/src -I../tbb_2020.3/src/rml/include -I../tbb_2020.3/include -I../tbb_2020.3/src/tbbmalloc -I../tbb_2020.3/src/tbbmalloc ../tbb_2020.3/src/tbbmalloc/backend.cpp
In file included from ../tbb_2020.3/include/tbb/tbb_config.h:36,
                 from ../tbb_2020.3/include/tbb/tbb_stddef.h:91,
                 from ../tbb_2020.3/src/tbbmalloc/Customize.h:21,
                 from ../tbb_2020.3/src/tbbmalloc/TypeDefinitions.h:54,
                 from ../tbb_2020.3/src/tbbmalloc/tbbmalloc_internal.h:21,
                 from ../tbb_2020.3/src/tbbmalloc/backend.cpp:19:
/usr/include/c++/10/cstddef:49:10: fatal error: bits/c++config.h: No such file or directory
   49 | #include <bits/c++config.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [/tmp/cmdstan-2.30.1/stan/lib/stan_math/lib/tbb_2020.3/build/Makefile.tbbmalloc:65: backend.o] Error 1
make[1]: Leaving directory '/tmp/cmdstan-2.30.1/stan/lib/stan_math/lib/tbb'
make: *** [stan/lib/stan_math/make/libraries:173: stan/lib/stan_math/lib/tbb/tbbmalloc.def] Error 2
make: *** Waiting for unfinished jobs....

this search implies that libstdc++-10-dev should provide it, but this does not avail me.

The cstddef file that triggers this lives in /usr/include/c++/10/, which does contain a bits folder but not with that file. The file it wants is in /usr/include/i386-linux-gnu/c++/10/bits.

this discussion looks relevant. It seems on Debian with this version of cmdstan we’re using the wrong stdlib.

I’ve tried to build tbb with clang instead of g++ with make compiler=clang but I get the same error, just from a different compiler. I can however change the behaviour by specifying make compiler=clang CXXFLAGS+=-stdlib=libc++ upon which now #include <cstddef> fails.

Any guidance would be appreciated!

2

I’ve gotten past this with the help of the debian package libtbb-dev (helpful link removed because it was considered spam); I downloaded tbb_2020.3.orig.tar.xz and tbb_2020.3-1.debian.tar.xz, and manually applied the tweak-64bits-hppa-alpha.patch (certainly unnecessary), linuxreleasedetect.patch, fixonlinux.patch, adddebug.patch, and buildi386.patch. I symlinked this patched source directory to stan/lib/stan_math/lib/tbb_2020.3 and was able to successfully compile that subcomponent.

I am now having an issue linking bin/cmdstan/stansummary.o: (.eh_frame+0x33cb): undefined reference to `__gxx_personality_v0’. I will try nuking my entire source tree and start from scratch with what I’ve learned so far.

3

That worked:

--- CmdStan v2.30.1 built ---

Here’s my attempt to document the required steps for anyone else encountering the same issue:

  1. Download the 2.30.1 release of CmdStan
  2. Download the orig and debian tarballs from Debian’s package site under the libtbb-dev package - the forum software won’t allow me to post the link, but it’s above.
  3. Extract tbb_2020.3-1.debian.tar.xz, keeping note of where the extracted debian directory is
  4. Extract tbb_2020.3.orig.tar.xz (obtained above) and cd into the extracted oneTBB-2020.3 directory
  5. Apply the following patches with this command (same order I did): patch -p1 < /PATH/TO/PATCH.patch: buildi386.patch, fixnonlinux.patch, linuxreleasedetect.patch, tweak-64bits-hppa-alpha.patch, adddebug.patch
  6. Rename the existing tbb_2020.3 directory at cmdstan-2.30.1/stan/lib/stan_math/lib
  7. Symlink your patched oneTBB-2020.3 directory to replace the above: ln -s /PATH/TO/oneTBB-2020.3 /PATH/TO/cmdstan-2.30.1/stan/lib/stan_math/lib/tbb_2020.3
  8. cd into your oneTBB-2020.3 directory and make the project. I had to do this from here, rather than allow CmdStan’s build system handle it
  9. If you’ve successfully built TBB, go back to the root cmdstan-2.30.1 directory and make build, which should work fine this time around.

My selection of patches is pretty arbitrary and I do not claim that is it precisely correct; I’m just a superstitious pigeon trying to make food appear here, sharing the dance that made that happen.

1 Like
4

Thank you @isosphere for this bug report and the details explanation of this fix!

5

I don’t think it’s necessary to use a sym link in this case, you should be able to set TBB_LIB and TBB_INC in your make/local file to point to TBB you downloaded

1 Like
6

Oh, that would be easier. Then Debian users can probably just install the libtbb-dev package using apt and direct those variables there and avoid the patching steps; I didn’t know that was an option!

1 Like
7

It’s not terribly well documented but we use it for the Conda package.

There is also a make variable TBB_INTERFACE_NEW which you can set to true to enable the use of oneTBB 2021+

1 Like