Stan on IBM Power: Compile and Linking Error

Interfaces
1

As mentioned in a previous post (see here), I am trying to run a COVID model (outlined here) on an IBM Power 9 system. I have been able to get stanc compiled for Power 9 and can now generate C code from a Stan file, but am now running into a number of errors during the “compile and linking” stage.

I am trying to build on a IBM Power 9 system using GCC 8.4.0, the attached text document* (error.txt) contains the lengthy error message.

Any pointers are appreciated!

2

I think the issue is that call to g++ isn’t using GCC 8.4. The errors in your output are typical of those when using a compiler that doesn’t fully support the c++14 spec.

Also, looking at these lines in the output:

/usr/include/c++/4.8.2/bits/stl_iterator.h:1104:5: note: template<class _IteratorL, class _IteratorR> decltype ((__x.base() - __y.base())) std::operator-(const std::move_iterator<_Iterator>&, const std::move_iterator<_IteratorR>&)
     operator-(const move_iterator<_IteratorL>& __x,

It looks like it’s using GCC 4.8.2. You can probably confirm by checking the return from:

g++ -v
2 Likes
3

Ah that was silly, I forgot to run module load before piping the error to a text file. After running module load the output of g++ -v is:

Reading specs from /opt/software/builder/developers/compilers/gcc/8.4.0/1/default/lib/gcc/powerpc64le-unknown-linux-gnu/8.4.0/specs
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/opt/software/builder/developers/compilers/gcc/8.4.0/1/default/libexec/gcc/powerpc64le-unknown-linux-gnu/8.4.0/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
Target: powerpc64le-unknown-linux-gnu
Configured with: ../gcc-8.4.0/configure --prefix=/opt/software/builder/developers/compilers/gcc/8.4.0/1/default --enable-shared --enable-threads=posix --with-system-zlib --enable-__cxa_atexit --enable-languages=c,c++,objc,obj-c++,fortran,go,lto --enable-offload-targets=nvptx-none=/opt/software/builder/developers/compilers/gcc/8.4.0/1/default/nvptx-none --with-cuda-driver=/opt/software/builder/developers/compilers/cuda/10.2.89/1/default/lib64
Thread model: posix
gcc version 8.4.0 (GCC)

And I now see a different set of errors

4

What versions of the stan compiler and cmdstan are you using? Can you use the version on develop (or 2.27?)

3 Likes
5

You hit the nail on the head! I was initially using Stan 2.23, moving to 2.27 allows me to build with no errors. Some warnings were produced (see below), but an executable is produced and runs. Thank you for the help!

--- Compiling, linking C++ code ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2020.3/include   -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include    -DBOOST_DISABLE_ASSERTS         -c -Wno-ignored-attributes   -x c++ -o ../src/models/deaths_and_111_calls.o ../src/models/deaths_and_111_calls.hpp
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2020.3/include   -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include    -DBOOST_DISABLE_ASSERTS               -Wl,-L,"/users/mjcarter/covid-19-uk/cmdstan-2.27.0/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/users/mjcarter/covid-19-uk/cmdstan-2.27.0/stan/lib/stan_math/lib/tbb"      ../src/models/deaths_and_111_calls.o src/cmdstan/main.o        -Wl,-L,"/users/mjcarter/covid-19-uk/cmdstan-2.27.0/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/users/mjcarter/covid-19-uk/cmdstan-2.27.0/stan/lib/stan_math/lib/tbb"   stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_kinsol.a  stan/lib/stan_math/lib/tbb/libtbb.so.2 -o ../src/models/deaths_and_111_calls
/usr/bin/ld: ../src/models/deaths_and_111_calls.o: In function `double boost::math::detail::digamma_imp<double, boost::integral_constant<int, 53>, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >(double, boost::integral_constant<int, 53> const*, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> const&) [clone .constprop.1946]':
deaths_and_111_calls.hpp:(.text+0x22a4): toc optimization is not supported for 0xe5890002 instruction.
/usr/bin/ld: deaths_and_111_calls.hpp:(.text+0x22b4): toc optimization is not supported for 0xe5a90002 instruction.
/usr/bin/ld: deaths_and_111_calls.hpp:(.text+0x22c8): toc optimization is not supported for 0xe4290002 instruction.
/usr/bin/ld: deaths_and_111_calls.hpp:(.text+0x22e0): toc optimization is not supported for 0xe4090002 instruction.
/usr/bin/ld: ../src/models/deaths_and_111_calls.o: In function `double boost::math::detail::gamma_imp<double, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy>, boost::math::lanczos::lanczos13m53>(double, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> const&, boost::math::lanczos::lanczos13m53 const&)':
deaths_and_111_calls.hpp:(.text._ZN5boost4math6detail9gamma_impIdNS0_8policies6policyINS3_10pole_errorILNS3_17error_policy_typeE1EEENS3_14overflow_errorILS6_1EEENS3_13promote_floatILb0EEENS3_14promote_doubleILb0EEENS3_14default_policyESE_SE_SE_SE_SE_SE_SE_SE_EENS0_7lanczos12lanczos13m53EEET_SI_RKT0_RKT1_[_ZN5boost4math6detail9gamma_impIdNS0_8policies6policyINS3_10pole_errorILNS3_17error_policy_typeE1EEENS3_14overflow_errorILS6_1EEENS3_13promote_floatILb0EEENS3_14promote_doubleILb0EEENS3_14default_policyESE_SE_SE_SE_SE_SE_SE_SE_EENS0_7lanczos12lanczos13m53EEET_SI_RKT0_RKT1_]+0x94): toc optimization is not supported for 0xe7e90002 instruction.
/usr/bin/ld: ../src/models/deaths_and_111_calls.o: In function `double boost::math::detail::lgamma_imp<double, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy>, boost::math::lanczos::lanczos13m53>(double, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> const&, boost::math::lanczos::lanczos13m53 const&, int*)':
deaths_and_111_calls.hpp:(.text._ZN5boost4math6detail10lgamma_impIdNS0_8policies6policyINS3_10pole_errorILNS3_17error_policy_typeE1EEENS3_14overflow_errorILS6_1EEENS3_13promote_floatILb0EEENS3_14promote_doubleILb0EEENS3_14default_policyESE_SE_SE_SE_SE_SE_SE_SE_EENS0_7lanczos12lanczos13m53EEET_SI_RKT0_RKT1_Pi[_ZN5boost4math6detail10lgamma_impIdNS0_8policies6policyINS3_10pole_errorILNS3_17error_policy_typeE1EEENS3_14overflow_errorILS6_1EEENS3_13promote_floatILb0EEENS3_14promote_doubleILb0EEENS3_14default_policyESE_SE_SE_SE_SE_SE_SE_SE_EENS0_7lanczos12lanczos13m53EEET_SI_RKT0_RKT1_Pi]+0x3dc): toc optimization is not supported for 0xe4290002 instruction.
/usr/bin/ld: deaths_and_111_calls.hpp:(.text._ZN5boost4math6detail10lgamma_impIdNS0_8policies6policyINS3_10pole_errorILNS3_17error_policy_typeE1EEENS3_14overflow_errorILS6_1EEENS3_13promote_floatILb0EEENS3_14promote_doubleILb0EEENS3_14default_policyESE_SE_SE_SE_SE_SE_SE_SE_EENS0_7lanczos12lanczos13m53EEET_SI_RKT0_RKT1_Pi[_ZN5boost4math6detail10lgamma_impIdNS0_8policies6policyINS3_10pole_errorILNS3_17error_policy_typeE1EEENS3_14overflow_errorILS6_1EEENS3_13promote_floatILb0EEENS3_14promote_doubleILb0EEENS3_14default_policyESE_SE_SE_SE_SE_SE_SE_SE_EENS0_7lanczos12lanczos13m53EEET_SI_RKT0_RKT1_Pi]+0x3ec): toc optimization is not supported for 0xe4090002 instruction.
/usr/bin/ld: src/cmdstan/main.o: In function `boost::math::tools::promote_args<double, float, float, float, float, float>::type boost::math::digamma<double, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >(double, boost::math::policies::policy<boost::math::policies::pole_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::overflow_error<(boost::math::policies::error_policy_type)1>, boost::math::policies::promote_float<false>, boost::math::policies::promote_double<false>, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> const&) [clone .isra.737]':
main.cpp:(.text.startup+0x1474): toc optimization is not supported for 0xe5890002 instruction.
/usr/bin/ld: main.cpp:(.text.startup+0x1484): toc optimization is not supported for 0xe5a90002 instruction.
/usr/bin/ld: main.cpp:(.text.startup+0x1498): toc optimization is not supported for 0xe4290002 instruction.
/usr/bin/ld: main.cpp:(.text.startup+0x14b0): toc optimization is not supported for 0xe4090002 instruction.
rm -f ../src/models/deaths_and_111_calls.o
1 Like
6

Nice! Googling it the warnings seem fine, though I’ve never worked on power 9 so glgl

https://www.google.com/search?channel=fs&client=ubuntu&q=toc+optimization+is+not+supported

1 Like