Linking to MKL doesn't seem to work

Interfaces
1

Operating System: RHEL 7
Interface Version: 2.17.1
Compiler/Toolkit: Intel/MKL

I basically just followed the instructions in the CmdStan user guide almost to the letter. Here’s my make/local:

CXX = icc

# MKLROOT is an environment variable defined by the Intel compiler module

CXXFLAGS += -I$(MKLROOT)/include -DEIGEN_USE_MKL_ALL
CXXFLAGS += -fp-model precise -fp-model source

LDLIBS += -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64
LDLIBS += -lmkl_core -lmkl_sequential -lpthread -lm -ldl

The only change I made to LDLIBS was to add -ldl, per the MKL link line advisor here: https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

(I had also tried building without -ldl, and it didn’t seem to make a difference.)

CmdStan appears to have been successfully built. I get --- CmdStan v2.17.1 built --- after running make build, and as far as I can tell, there were no errors.

However, when it came to building the Bernoulli example, I got this:

--- Translating Stan model to C++ code ---
bin/stanc  examples/bernoulli/bernoulli.stan --o=examples/bernoulli/bernoulli.hpp
Model name=bernoulli_model
Input file=examples/bernoulli/bernoulli.stan
Output file=examples/bernoulli/bernoulli.hpp
Compiling pre-compiled header
icc -Wall -I . -isystem stan/lib/stan_math/lib/eigen_3.3.3 -isystem stan/lib/stan_math/lib/boost_1.64.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -std=c++1y -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -Wno-unused-function -Wno-uninitialized -I src -isystem stan/src -isystem stan/lib/stan_math/ -DFUSION_MAX_VECTOR_SIZE=12 -Wno-unused-local-typedefs -DEIGEN_NO_DEBUG -I/p/app/intel/l_psxe_2017.1.043/compilers_and_libraries_2017.1.132/linux/mkl/include -DEIGEN_USE_MKL_ALL -fp-model precise -fp-model source -DNO_FPRINTF_OUTPUT -pipe  -c -O3 stan/src/stan/model/model_header.hpp -o stan/src/stan/model/model_header.hpp.gch
icc: command line warning #10006: ignoring unknown option '-Wno-unused-local-typedefs'
icc: warning #10145: no action performed for file 'stan/src/stan/model/model_header.hpp'

--- Linking C++ model ---
icc -Wall -I . -isystem stan/lib/stan_math/lib/eigen_3.3.3 -isystem stan/lib/stan_math/lib/boost_1.64.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -std=c++1y -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -Wno-unused-function -Wno-uninitialized -I src -isystem stan/src -isystem stan/lib/stan_math/ -DFUSION_MAX_VECTOR_SIZE=12 -Wno-unused-local-typedefs -DEIGEN_NO_DEBUG -I/p/app/intel/l_psxe_2017.1.043/compilers_and_libraries_2017.1.132/linux/mkl/include -DEIGEN_USE_MKL_ALL -fp-model precise -fp-model source -DNO_FPRINTF_OUTPUT -pipe    -O3 -o examples/bernoulli/bernoulli src/cmdstan/main.cpp -include examples/bernoulli/bernoulli.hpp stan/lib/stan_math/lib/cvodes_2.9.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/cvodes_2.9.0/lib/libsundials_cvodes.a
icc: command line warning #10006: ignoring unknown option '-Wno-unused-local-typedefs'
/tmp/icc58NWAj.o: In function `void Eigen::internal::generic_product_impl<Eigen::Product<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double, double>, Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, 1, -1, 1, 1, -1> const> const, Eigen::Transpose<Eigen::Matrix<double, -1, 1, 0, -1, 1> > const>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0>, Eigen::Matrix<double, -1, 1, 0, -1, 1>, Eigen::DenseShape, Eigen::DenseShape, 6>::evalTo<Eigen::Matrix<double, 1, 1, 0, 1, 1> >(Eigen::Matrix<double, 1, 1, 0, 1, 1>&, Eigen::Product<Eigen::CwiseBinaryOp<Eigen::internal::scalar_product_op<double, double>, Eigen::CwiseNullaryOp<Eigen::internal::scalar_constant_op<double>, Eigen::Matrix<double, 1, -1, 1, 1, -1> const> const, Eigen::Transpose<Eigen::Matrix<double, -1, 1, 0, -1, 1> > const>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0> const&, Eigen::Matrix<double, -1, 1, 0, -1, 1> const&)':
main.cpp:(.text._ZN5Eigen8internal20generic_product_implINS_7ProductINS_13CwiseBinaryOpINS0_17scalar_product_opIddEEKNS_14CwiseNullaryOpINS0_18scalar_constant_opIdEEKNS_6MatrixIdLi1ELin1ELi1ELi1ELin1EEEEEKNS_9TransposeINS9_IdLin1ELi1ELi0ELin1ELi1EEEEEEENS9_IdLin1ELin1ELi0ELin1ELin1EEELi0EEESF_NS_10DenseShapeESL_Li6EE6evalToINS9_IdLi1ELi1ELi0ELi1ELi1EEEEEvRT_RKSK_RKSF_[_ZN5Eigen8internal20generic_product_implINS_7ProductINS_13CwiseBinaryOpINS0_17scalar_product_opIddEEKNS_14CwiseNullaryOpINS0_18scalar_constant_opIdEEKNS_6MatrixIdLi1ELin1ELi1ELi1ELin1EEEEEKNS_9TransposeINS9_IdLin1ELi1ELi0ELin1ELi1EEEEEEENS9_IdLin1ELin1ELi0ELin1ELin1EEELi0EEESF_NS_10DenseShapeESL_Li6EE6evalToINS9_IdLi1ELi1ELi0ELi1ELi1EEEEEvRT_RKSK_RKSF_]+0x26f): undefined reference to `dgemv_'

There were other undefined references, such as dgemm_, vdSqrt, vdSqr, and LAPACKE_dsyev.

2

You may have to do CXX=icpc. AFAIK, no one has tried Intel stuff in a while. @wds15 used to. You can search for icc or icpc on
https://groups.google.com/forum/#!forum/stan-dev
to see some old threads. It did compile (slowly) and run (at the same speed as g++ / clang++) at one point.

3

Using icpc rather than icc doesn’t make a difference.

I do notice that I don’t see anything from LDLIBS on the link line. I tried assuming that the CmdStan manual was wrong and changed LDLIBS to LDFLAGS, and that at least got the Bernoulli example to compile. That suggests that either the manual or the makefiles need changing.

3 Likes
4

I think you’re right. I think the LDLIBS isn’t being used by CmdStan for whatever reason. I’ll update the doc.

1 Like