Can't compile exemple/bernoulli model after adding threading flags to CmdStan [Ubuntu 18.04.3]

Interfaces CmdStan
1

Hi everyone,

I am trying to implement threading on my local machine. Everything works fine with CmdStan. I can compile models, sample and use stansummary etc.

However, as soon as I had the flags:

echo "CXXFLAGS += -DSTAN_THREADS" > make/local
echo "CXXFLAGS += -pthread" >> make/local

CmdStan doesn’t compile models, with the following ouput for the examples/bernoulli/bernoulli.stan (same error output for McElreath map_rect turorial)

--- Translating Stan model to C++ code ---
bin/stanc  --o=examples/bernoulli/bernoulli.hpp examples/bernoulli/bernoulli.stan
Model name=bernoulli_model
Input file=examples/bernoulli/bernoulli.stan
Output file=examples/bernoulli/bernoulli.hpp

--- Compiling, linking C++ code ---
g++ -DSTAN_THREADS -pthread -std=c++1y -pthread -Wno-sign-compare     -O3 -I src -I stan/src -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.3 -I stan/lib/stan_math/lib/boost_1.69.0 -I stan/lib/stan_math/lib/sundials_4.1.0/include    -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION     -c  -x c++ -o examples/bernoulli/bernoulli.o examples/bernoulli/bernoulli.hpp
g++ -DSTAN_THREADS -pthread -std=c++1y -pthread -Wno-sign-compare     -O3 -I src -I stan/src -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.3 -I stan/lib/stan_math/lib/boost_1.69.0 -I stan/lib/stan_math/lib/sundials_4.1.0/include    -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION            examples/bernoulli/bernoulli.o src/cmdstan/main.o        stan/lib/stan_math/lib/sundials_4.1.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_4.1.0/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_4.1.0/lib/libsundials_idas.a  -o examples/bernoulli/bernoulli
/usr/bin/ld: _ZN4stan4math22AutodiffStackSingletonINS0_4variENS0_15chainable_allocEE9instance_E: TLS definition in examples/bernoulli/bernoulli.o section .tbss._ZN4stan4math22AutodiffStackSingletonINS0_4variENS0_15chainable_allocEE9instance_E[_ZN4stan4math22AutodiffStackSingletonINS0_4variENS0_15chainable_allocEE9instance_E] mismatches non-TLS reference in src/cmdstan/main.o
src/cmdstan/main.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make/program:38: recipe for target 'examples/bernoulli/bernoulli' failed
make: *** [examples/bernoulli/bernoulli] Error 1

Could there be some g++ depencies missing?

Reinstalling CmdStan does not solve this issue.

Thank you for your help!

Please provide this additional information in addition to your question:

  • Operating System: Ubuntu 18.04.3
  • CmdStan Version: 2.20.0
  • Compiler/Toolkit: g++
2

I believe this is due to the link order bug we introduced in the 2.20 release. Apologies. This has since been fixed with https://github.com/stan-dev/cmdstan/pull/717

I think you have 3 options to fix this, depending on what you feel comfortable with:

  • clone the develop branch of Cmdstan from Github
    git clone --recurse-submodules https://github.com/stan-dev/cmdstan.git
  • use the 2.20 release you have and change line 41 in the make/program file to
    $(LINK.cpp) $*.o $(CMDSTAN_MAIN_O) $(LDLIBS) $(LIBSUNDIALS) $(MPI_TARGETS) $(OUTPUT_OPTION)
  • use Cmdstan 2.19.1
1 Like
3

Did you make a

make clean-all

before trying out the threading stuff? You need to start of a clean install and what you report sounds like things are not clean.

1 Like
4

Thank you very much @rok_cesnovar and @wds15.

Doing a make clean-all before including the flags, solved the issue for me.

Then I just had to make build, then build the model and everything is working now. It works both for models with and without threading.

Thanks again.

5

In my situation, under Linux, I have to make clean-all, then

echo "CXXFLAGS += -DSTAN_THREADS" > make/local
echo "CXXFLAGS += -pthread" >> make/local

And rebuild the cmdstan, with make build -j4 under the cmdstan directory.

After reinstalling cmdstan, I can build my model and use multi-thread support by setting export STAN_NUM_THREADS=-1 (using all the cpus I have).

But on mac, I have no such problem. I just need to add

echo "CXXFLAGS += -DSTAN_THREADS" > make/local

before compiling my model.