Cmdstan binaries for unit testing

I am working on (yet another) Julia wrapper for cmdstan. The user would need to install it separately to use the wrapper, but I am not sure how to unit test this setup.

I am using Travis, and I could make the test script download and compile cmdstan each time it is run, but this is time-consuming and an unnecessary waste of resources. So I am wondering if there are (unofficial) linux binary distributions for cmdstan that I could just grab for automated testing, eg as a byproduct of a buildbot.

Not that I know of, but @seantalts should know. The problem I foresee is that they’d have to be binary compatible with whatever else you have.

As far as I know, the only thing that gets compiled in CmdStan is the parser (and maybe the CVODES library for ODE solving). I think you’ll find if you cut the optimization level down, it’ll go a lot faster rebuilding while you test. We can’t do that on Windows because the toolchain blows up with too much debug info at -O0 optimization levels, but it’s no problem on the Mac or Linux.

Figured it out: Travis can cache the CmdStan repo and compiled stanc. Relevant lines from .travis.yml:

cache:
  directories:
  - $HOME/cmdstan

will cache the directory, and

script:
  - if [[ -a .git/shallow ]]; then git fetch --unshallow; fi
  - cd $HOME; if [[ ! -d cmdstan ]]; then git clone https://github.com/stan-dev/cmdstan.git; fi
  - cd $HOME/cmdstan; git checkout v2.15.0; make stan-update; make build
  - export CMDSTAN_HOME=$HOME/cmdstan
  - cd $TRAVIS_BUILD_DIR

will pull and build it for the first time (and only then). Should be followed by repo-specific test commands.

3 Likes

Thanks for posting the solution. Did you think we should be adding more caching to the testing of some repos on stan-dev?

That depends — in case the cacheable stuff is significant, I would consider it. I mainly did it because for me compiling CmdStan and the dependencies dominated test runtimes.

Whoa. Looks like caching is new! https://docs.travis-ci.com/user/caching/