Linux compile with gcc-6.3: possible?

[pauljohn32]

I’ve not been able to compile rstan for a while, but am making another run at it today. I’ve tried every few months…

On the cluster computing system, we have RedHat 6 with various package updates. The R is Microsoft R and the gcc I’ve got is 6.3. The toolchain test on (C++14 standard requested but CXX14 is not defined - #3 by Jean_Billie) succeeds.

I’ve replaced ~/.R/Makevars with the latest advice I find in your page:

$ cat ~/.R/Makevars
CXX14FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -fPIC
CXX14 = g++ # could also you clang++ but your compiler may have a version number postfix


CXX14FLAGS+=-flto -Wno-unused-local-typedefs -Wno-ignored-attributes -Wno-deprecated-declarations

I’ve already got Rcpp configured, works OK.

install.packages(“rstan”, type = “source”, lib = TARGETLIB)

Runs for a while, fails:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  814k  100  814k    0     0  28.5M      0 --:--:-- --:--:-- --:--:-- 39.7M
* installing *source* package ‘rstan’ ...
** package ‘rstan’ successfully unpacked and MD5 sums checked
** libs
g++ -std=c++1y  -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include    -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION      -c chains.cpp -o chains.o
g++ -std=c++1y  -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include    -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION      -c init.cpp -o init.o
g++ -std=c++1y  -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include    -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION      -c lang__ast_def.cpp -o lang__ast_def.o
g++ -std=c++1y  -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include    -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION      -c lang__grammars__bare_type_grammar_inst.cpp -o lang__grammars__bare_type_grammar_inst.o
g++ -std=c++1y  -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include    -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION      -c lang__grammars__expression07_grammar_inst.cpp -o lang__grammars__expression07_grammar_inst.o
In file included from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Core:388:0,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Dense:1,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/mean.hpp:5,
                 from chains.cpp:19:
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:60:39: warning: ignoring attributes on template argument ‘__m128 {aka __vector(4) float}’ [-Wignored-attributes]
 template<> struct is_arithmetic<__m128>  { enum { value = true }; };
                                       ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:61:40: warning: ignoring attributes on template argument ‘__m128i {aka __vector(2) long long int}’ [-Wignored-attributes]
 template<> struct is_arithmetic<__m128i> { enum { value = true }; };
                                        ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:62:40: warning: ignoring attributes on template argument ‘__m128d {aka __vector(2) double}’ [-Wignored-attributes]
 template<> struct is_arithmetic<__m128d> { enum { value = true }; };
                                        ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:161:43: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4f {aka __vector(4) float}’ [-Wignored-attributes]
 template<> struct unpacket_traits<Packet4f> { typedef float  type; enum {size=4, alignment=Aligned16}; typedef Packet4f half; };
                                           ^
In file included from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Core:388:0,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Dense:1,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/mean.hpp:5,
                 from chains.cpp:19:
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:162:43: warning: ignoring attributes on template argument ‘Eigen::internal::Packet2d {aka __vector(2) double}’ [-Wignored-attributes]
 template<> struct unpacket_traits<Packet2d> { typedef double type; enum {size=2, alignment=Aligned16}; typedef Packet2d half; };
                                           ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:163:43: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4i {aka __vector(2) long long int}’ [-Wignored-attributes]
 template<> struct unpacket_traits<Packet4i> { typedef int    type; enum {size=4, alignment=Aligned16}; typedef Packet4i half; };
                                           ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:712:35: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4f {aka __vector(4) float}’ [-Wignored-attributes]
 struct palign_impl<Offset,Packet4f>
                                   ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:735:35: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4i {aka __vector(2) long long int}’ [-Wignored-attributes]
 struct palign_impl<Offset,Packet4i>
                                   ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:758:35: warning: ignoring attributes on template argument ‘Eigen::internal::Packet2d {aka __vector(2) double}’ [-Wignored-attributes]
 struct palign_impl<Offset,Packet2d>
                                   ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:772:34: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4f {aka __vector(4) float}’ [-Wignored-attributes]
 ptranspose(PacketBlock<Packet4f,4>& kernel) {
                                  ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:777:34: warning: ignoring attributes on template argument ‘Eigen::internal::Packet2d {aka __vector(2) double}’ [-Wignored-attributes]
 ptranspose(PacketBlock<Packet2d,2>& kernel) {
                                  ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/PacketMath.h:784:34: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4i {aka __vector(2) long long int}’ [-Wignored-attributes]
 ptranspose(PacketBlock<Packet4i,4>& kernel) {
                                  ^
In file included from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Core:371:0,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Dense:1,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/mean.hpp:5,
                 from chains.cpp:19:
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/Default/ConjHelper.h:15:70: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4f {aka __vector(4) float}’ [-Wignored-attributes]
   template<> struct conj_helper<PACKET_REAL, PACKET_CPLX, false,false> {                                          \
                                                                      ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/Complex.h:232:1: note: in expansion of macro ‘EIGEN_MAKE_CONJ_HELPER_CPLX_REAL’
 EIGEN_MAKE_CONJ_HELPER_CPLX_REAL(Packet2cf,Packet4f)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/Default/ConjHelper.h:22:70: warning: ignoring attributes on template argument ‘Eigen::internal::Packet4f {aka __vector(4) float}’ [-Wignored-attributes]
   template<> struct conj_helper<PACKET_CPLX, PACKET_REAL, false,false> {                                          \
                                                                      ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/Complex.h:232:1: note: in expansion of macro ‘EIGEN_MAKE_CONJ_HELPER_CPLX_REAL’
 EIGEN_MAKE_CONJ_HELPER_CPLX_REAL(Packet2cf,Packet4f)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/Default/ConjHelper.h:15:70: warning: ignoring attributes on template argument ‘Eigen::internal::Packet2d {aka __vector(2) double}’ [-Wignored-attributes]
   template<> struct conj_helper<PACKET_REAL, PACKET_CPLX, false,false> {                                          \
                                                                      ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/Complex.h:417:1: note: in expansion of macro ‘EIGEN_MAKE_CONJ_HELPER_CPLX_REAL’
 EIGEN_MAKE_CONJ_HELPER_CPLX_REAL(Packet1cd,Packet2d)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/Default/ConjHelper.h:22:70: warning: ignoring attributes on template argument ‘Eigen::internal::Packet2d {aka __vector(2) double}’ [-Wignored-attributes]
   template<> struct conj_helper<PACKET_CPLX, PACKET_REAL, false,false> {                                          \
                                                                      ^
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/arch/SSE/Complex.h:417:1: note: in expansion of macro ‘EIGEN_MAKE_CONJ_HELPER_CPLX_REAL’
 EIGEN_MAKE_CONJ_HELPER_CPLX_REAL(Packet1cd,Packet2d)
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Core:364:0,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/Dense:1,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4,
                 from /panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include/stan/math/prim/mat/fun/mean.hpp:5,
                 from chains.cpp:19:
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/util/XprHelper.h: In instantiation of ‘struct Eigen::internal::find_best_packet<float, 4>’:
/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include/Eigen/src/Core/Matrix.h:22:57:   required from ‘struct Eigen::internal::traits<Eigen::Matrix<float, 4, 1> >’

[snip]


g++   -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include -DNDEBUG -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION     -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -fPIC -flto -Wno-unused-local-typedefs -Wno-ignored-attributes -Wno-deprecated-declarations -c sparse_extractors.cpp -o sparse_extractors.o
g++   -I/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/include -DNDEBUG -I"../inst/include" -I"`"/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/Rcpp/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/RcppEigen/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/BH/include" -I"/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/StanHeaders/include" -DU_STATIC_IMPLEMENTATION     -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -fPIC -flto -Wno-unused-local-typedefs -Wno-ignored-attributes -Wno-deprecated-declarations -c stanc.cpp -o stanc.o
g++ -shared -L/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/lib -o rstan.so chains.o init.o lang__ast_def.o lang__grammars__bare_type_grammar_inst.o lang__grammars__expression07_grammar_inst.o lang__grammars__expression_grammar_inst.o lang__grammars__functions_grammar_inst.o lang__grammars__indexes_grammar_inst.o lang__grammars__program_grammar_inst.o lang__grammars__semantic_actions.o lang__grammars__statement_2_grammar_inst.o lang__grammars__statement_grammar_inst.o lang__grammars__term_grammar_inst.o lang__grammars__var_deccls_grammar_inst.o lang__grammars__whitespace_grammar_inst.o misc.o pointer-tools.o sparse_extractors.o stanc.o -L/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/lib -lR
lto1: internal compiler error: in add_symbol_to_partition_1, at lto/lto-partition.c:202
0x5fe7fb add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:202
0x5fe5fd add_references_to_partition
	../../gcc-6.3.0/gcc/lto/lto-partition.c:92
0x5fe7a4 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:175
0x5fe5fd add_references_to_partition
	../../gcc-6.3.0/gcc/lto/lto-partition.c:92
0x5fe7a4 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:175
0x5fe8b3 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:160
0x5fe8b3 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:160
0x5fe8b3 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:160
0x5fe8b3 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:160
0x5fe8b3 add_symbol_to_partition_1
	../../gcc-6.3.0/gcc/lto/lto-partition.c:160
0x5ff1e3 lto_balanced_map(int)
	../../gcc-6.3.0/gcc/lto/lto-partition.c:546
0x5f86d8 do_whole_program_analysis
	../../gcc-6.3.0/gcc/lto/lto.c:3128
0x5f86d8 lto_main()
	../../gcc-6.3.0/gcc/lto/lto.c:3322
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: fatal error: g++ returned 1 exit status
compilation terminated.
collect2: fatal error: lto-wrapper returned 1 exit status
compilation terminated.
make: *** [rstan.so] Error 1
ERROR: compilation failed for package ‘rstan’
* removing ‘/panfs/pfs.local/work/crmda/tools/mro/3.5/site-library/rstan’

The downloaded source packages are in
	‘/library/tmp/Rtmp0f1wU2/downloaded_packages’
Warning message:
In install.packages("rstan", type = "source", lib = TARGETLIB) :
  installation of package ‘rstan’ had non-zero exit status
>

[bgoodri]

@pauljohn32

lto1: internal compiler error: in add_symbol_to_partition_1, at lto/lto-partition.c:202

Try it without the -flto in CXX14FLAGS in ~/.R/Makevars

[pauljohn32]

Success after that change. Thanks

[bgoodri]

It appears as if the underlying compiler bug was fixed for 8.1 but has not been backported

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69866

And 6.5 was the last release on the g++-6 series a few days ago. LTO is not critical, but there are some benchmarks supporting it

thinLTO / standard benchmarks?

I’ve been using @seantalts 's code in the “perf” branch of cmdstan to do some performance tests, and it seems like thinLTO (a clang feature that tries to get most of the benefit of LTO without impacting compile-time) gives a noticable improvement. Since I’m new around here, my questions are: What’s the standard of evidence used to evaluate performance changes? Is there a standard set of benchmarks that should be run? Using @seantalts 's scripts, here’s the ratio of runtimes for a variety of …

You may be able to install rstan without the -flto flag but add it back to compile Stan programs at runtime.