Clang++ include path order


#1

Anyone (@bgoodri) know where to find the include order when using -I or -isystem?

I’m running into weird behavior. I had read the clang documentation to mean that I could use them interchangeably with -isystem just suppressing warnings, but I’m finding that by using -isystem it’s including it after system paths (which aren’t explicitly passed to the compiler).

Just hoping to find some doc for now. I’m clearly not searching for the right things.


#2

I assume that clang follows gcc on this
https://gcc.gnu.org/onlinedocs/gcc-7.2.0/gcc/Directory-Options.html


#3

@bgoodri, thanks!

Compilers don’t follow the doc

That’s the doc that I found and for whatever reason, it looks like clang++ doesn’t actually respect the lookup order described in that doc (neither does g++ on Mac). I’ve quoted the relevant part of it here:

  1. For the quote form of the include directive, the directory of the current file is searched first.
  2. For the quote form of the include directive, the directories specified by -iquote options are searched in left-to-right order, as they appear on the command line.
  3. Directories specified with -I options are scanned in left-to-right order.
  4. Directories specified with -isystem options are scanned in left-to-right order.
  5. Standard system directories are scanned.
  6. Directories specified with -idirafter options are scanned in left-to-right order.

I’m getting compiler warnings from Boost. The warnings are coming from where it’s installed on the system, not in the directory I specified using -isystem. If I change -isystem to -I, it finds the correct directory. The real issue isn’t with the warning, but what gets included. At the very least, it needs to be consistent.

Why it matters

The reason this is (going to be) an issue is because I got to this state by installing Boost using their tools. This is one way to get the runtime libraries for Boost MPI. Another is to install from Homebrew or some other package manager, but I presume the issue is the same. There’s a good chance we’d include Boost from the wrong path, which could lead to some really hard-to-track bugs.

Below are two calls to clang++. The top one uses -isystem lib/boost_1.65.1 and the bottom one replaces that with -I lib/boost_1.65.1. You’ll see that the warnings in the top come from /usr/local/include/boost and the bottom from lib/boost_1.65.1 (the correct place).

Include specified as: -isystem lib/boost_1.65.1

> clang++ -Wall -I . -isystem lib/eigen_3.3.3 -isystem lib/boost_1.65.1 -isystem lib/cvodes_2.9.0/include -std=c++1y -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -Wno-unused-function -Wno-uninitialized -stdlib=libc++ -Wno-unknown-warning-option -Wno-tautological-compare -Wsign-compare -DGTEST_USE_OWN_TR1_TUPLE -isystem lib/gtest_1.7.0/include -isystem lib/gtest_1.7.0 -O3 -DNO_FPRINTF_OUTPUT -pipe  -c -o test/unit/math_include_test.o test/unit/math_include_test.cpp -Wno-system-headers
In file included from test/unit/math_include_test.cpp:1:
In file included from ./stan/math.hpp:4:
In file included from ./stan/math/rev/mat.hpp:12:
In file included from ./stan/math/prim/mat.hpp:307:
In file included from ./stan/math/prim/arr.hpp:38:
In file included from ./stan/math/prim/arr/functor/integrate_ode_rk45.hpp:17:
In file included from /usr/local/include/boost/numeric/odeint.hpp:61:
In file included from /usr/local/include/boost/numeric/odeint/util/multi_array_adaption.hpp:29:
In file included from /usr/local/include/boost/multi_array.hpp:21:
In file included from /usr/local/include/boost/multi_array/base.hpp:28:
/usr/local/include/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
      typedef typename Array::index_range index_range;
                                          ^
/usr/local/include/boost/multi_array/concept_checks.hpp:43:37: warning: unused typedef 'index' [-Wunused-local-typedef]
      typedef typename Array::index index;
                                    ^
/usr/local/include/boost/multi_array/concept_checks.hpp:53:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
      typedef typename Array::index_range index_range;
                                          ^
/usr/local/include/boost/multi_array/concept_checks.hpp:54:37: warning: unused typedef 'index' [-Wunused-local-typedef]
      typedef typename Array::index index;
                                    ^
4 warnings generated.

Include specified as: -I lib/boost_1.65.1

clang++ -Wall -I . -isystem lib/eigen_3.3.3 -I lib/boost_1.65.1 -isystem lib/cvodes_2.9.0/include -std=c++1y -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -Wno-unused-function -Wno-uninitialized -stdlib=libc++ -Wno-unknown-warning-option -Wno-tautological-compare -Wsign-compare -DGTEST_USE_OWN_TR1_TUPLE -isystem lib/gtest_1.7.0/include -isystem lib/gtest_1.7.0 -O3 -DNO_FPRINTF_OUTPUT -pipe  -c -o test/unit/math_include_test.o test/unit/math_include_test.cpp -Wno-system-headers
In file included from test/unit/math_include_test.cpp:1:
In file included from ./stan/math.hpp:4:
In file included from ./stan/math/rev/mat.hpp:12:
In file included from ./stan/math/prim/mat.hpp:307:
In file included from ./stan/math/prim/arr.hpp:38:
In file included from ./stan/math/prim/arr/functor/integrate_ode_rk45.hpp:17:
In file included from lib/boost_1.65.1/boost/numeric/odeint.hpp:61:
In file included from lib/boost_1.65.1/boost/numeric/odeint/util/multi_array_adaption.hpp:29:
In file included from lib/boost_1.65.1/boost/multi_array.hpp:21:
In file included from lib/boost_1.65.1/boost/multi_array/base.hpp:28:
lib/boost_1.65.1/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
      typedef typename Array::index_range index_range;
                                          ^
lib/boost_1.65.1/boost/multi_array/concept_checks.hpp:43:37: warning: unused typedef 'index' [-Wunused-local-typedef]
      typedef typename Array::index index;
                                    ^
lib/boost_1.65.1/boost/multi_array/concept_checks.hpp:53:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
      typedef typename Array::index_range index_range;
                                          ^
lib/boost_1.65.1/boost/multi_array/concept_checks.hpp:54:37: warning: unused typedef 'index' [-Wunused-local-typedef]
      typedef typename Array::index index;
                                    ^
4 warnings generated.

Proposed fix

For now, I’m going to propose changing -isystem to -I. Documentation says to use -isystem for vendor header files, but we’ve got to deal with compiler behavior, not compiler doc.

I’ll also suggest we add a few more flags to ignore warnings.

Any objections?


My clang++ version:

$ clang++ -v
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin17.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

#4

Not from me. Thanks for sorting through all this. What a pain!