As a followup to:
it would be good to have some guidance of the settings to be used with travisci and c++14. rstanarm currently cannot serve as an example, because it fails on travis for some other reason. My own attempts showed moderate progress only:
We haven’t been able to get rstanarm to work well on Travis for other reasons. But basically, you just need a toolchain that supports C++14 and the Linux configuration
https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Mac-or-Linux#config
“Basically” it’s no big problem, I have build rstanarm-like packages on travis since more than a year. More recently, I cannot get rid of this one:
Error in .shlib_internal(args) :
C++14 standard requested but CXX14 is not defined
* removing ‘/home/travis/R/Library/rstan’
Installation failed: Command failed (1)
SYou just need to execute this
dotR <- file.path(Sys.getenv("HOME"), ".R")
if (!file.exists(dotR)) dir.create(dotR)
MAKEVARS <- file.path(dotR, "Makevars")
if (!file.exists(MAKEVARS)) file.create(MAKEVARS)
cat(
"\nCXX14FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-macro-redefined",
"CXX14 = g++", # could also you clang++ but your compiler may have a version number postfix
file = MAKEVARS,
sep = "\n",
append = TRUE
)
or the equivalent so that CXX14 gets defined (although I would have thought it already is if you are using R >= 3.4.0).
This is the equivalent of http://dirk.eddelbuettel.com/blog/2017/06/13/ I had used before, but which was no longer necessary at some point. Back to the future.
I give up with , too much time lost trying variants (CRAN submission no problem). In case someone would like to give it a try:
and the error message is:
https://travis-ci.org/dmenne/breathteststan/jobs/443094844
g++ -std=c++1y -shared -L/home/travis/R-bin/lib -o rstan.so chains.o init.o lang__ast_def.o lang__grammars__bare_type_grammar_inst.o lang__grammars__expression07_grammar_inst.o lang__grammars__expression_grammar_inst.o lang__grammars__functions_grammar_inst.o lang__grammars__indexes_grammar_inst.o lang__grammars__program_grammar_inst.o lang__grammars__semantic_actions.o lang__grammars__statement_2_grammar_inst.o lang__grammars__statement_grammar_inst.o lang__grammars__term_grammar_inst.o lang__grammars__var_deccls_grammar_inst.o lang__grammars__whitespace_grammar_inst.o misc.o pointer-tools.o sparse_extractors.o stanc.o
/usr/bin/ld: chains.o: relocation R_X86_64_32 against `_ZGVZ7dataptrP7SEXPRECE3fun’ can not be used when making a shared object; recompile with -fPIC
chains.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make: *** [rstan.so] Error 1
I used roughly the same travis.yml for ctsem, and get the same error.
Good to know. What happens when you build locally on Windows or Linux?
Linux was smooth, after a few hours of struggle with things yesterday windows is ok too. Winbuilder is failing:
Wrote C++ file "stan_files/ctsm.cc"
d:/Compiler/gcc-4.9.3/mingw_32/bin/g++ -std=gnu++14 -I"D:/RCompile/recent/R/include" -DNDEBUG -I"../inst/include" -I"`"D:/RCompile/recent/R/bin/i386/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -I"`"D:/RCompile/recent/R/bin/i386/Rscript" --vanilla -e "cat(system.file('include', 'boost_not_in_BH', package = 'rstan'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DBOOST_NO_CXX11_RVALUE_REFERENCES -I"d:/RCompile/CRANpkg/lib/3.6/StanHeaders/include" -I"d:/RCompile/CRANguest/R-devel/lib/rstan/include" -I"d:/RCompile/CRANpkg/lib/3.6/BH/include" -I"d:/RCompile/CRANpkg/lib/3.6/Rcpp/include" -I"d:/RCompile/CRANpkg/lib/3.6/RcppEigen/include" -I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall -mtune=core2 -c stan_files/ctsm.cc -o stan_files/ctsm.o
cc1plus.exe: out of memory allocating 15002944 bytes
make: *** [stan_files/ctsm.o] Error 1
rm stan_files/ctsm.cc
but I’m going to try a submit to CRAN shortly anyway…
OK, so I, err, did what the error message told me to do, and rstan installs ;)
the -fPIC argument is needed. See here: https://stackoverflow.com/questions/5311515/gcc-fpic-option for discussion of fPIC.
before_install:
- mkdir -p ~/.R
- echo "CXX14FLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-macro-redefined" >> ~/.R/Makevars
- echo "CXX14=g++ -std=c++1y -fPIC" >> ~/.R/MakevarsOn Travis, you probably also want to set the -flto=thin flag.
@Charles_Driver: everything green with fPIC. I had seen that message but was somehow lost in compilation.
@boogri: flto=thin fails
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-receivedDo you have a link to the Travis log?
This is from my basic teststan package with flto = thin
https://api.travis-ci.org/v3/job/443243139/log.txt
Message is a different one for breathteststan:
https://api.travis-ci.org/v3/job/443260435/log.txt
Note: without flto, build is fine:
https://api.travis-ci.org/v3/job/443223868/log.txt
Switching back to the “no-flto” version gave an error, probably some memory problem;
https://api.travis-ci.org/v3/job/443267930/log.txt
I will try again tomorrow morning (CET), when load is lower on Travis. I had similar errors before.
Both with and without flto works during off-peak time in travis - nothing changed from yesterday’s on-peak failed build. I wish they would create better error messages when something fails because of technical limitations with the system.
I tried several times this morning to restart. With unchanged code, the success rate is absolutely erratic. Most frequently “the job did not respond withing 10 minutes”, but there are cases where logging end in the middle of nowhere without any error message. Build times run from 13 to 48 minutes. There is some indication that with flto there are more failures, possibly because optimization needs time.
Looks like the free version of Travis cannot reliably handle rstan because of timeout and memory constraints.
A bit disappointing but good to know… is it maybe worth filing an issue with Travis or do you think / know they are aware?
It’s difficult to created a stripped-down example that shows it. My minimal https://github.com/dmenne/teststan already needs 18 minutes in the early morning CET. I fear they don’t love these long-running compiles in the free scheme anyway; I can understand that, they have to make some $$$.
gitlab might provide an altnative, because they allow for a docker base image, but I could not get it to work yet, still “searching for runner” (they are all in Hawaii nowadays playing Ironmen).
As an alternative, I have set up a gitlab CI, linking to GitHub. Too bad, it’s a feature that will no longer be available after March 2019.
To celebrate this release, we’re offering this feature promotionally as part of GitLab.com Free through March 2019.
GitLab wants to get the full share of GitHub, so you can continue to use it when you keep your files on a public GitLab repository
This is much faster than TravisCI (9 minutes compared to 20-Inf) , because it uses a prebuild docker image dmenne/stanverse, child of rocker/shiny. Not the full “start from scratch approach”, but that anyway will be done on winbuilder/r-hub.
https://gitlab.com/dmenne/breathteststan/blob/master/.gitlab-ci.yml