Lately I have been running against many walls when messing with compilers/make/other system related stuff. At the end I was forced to install a VirtualBox on my machine to be able to reproduce errors happening on other systems to make progress.
From this experience I really think that we should make the effort and turn all our testing environments into some docker based thing. The thing is that I am not really experienced with docker. Thus, if people agree that we should do this (I really think so), I am calling out here if people have
- cmdstan docker images suitable as basis for our testing for mac and for linux
- we may consider to use the Travis docker images (but they use a really old 14.04 ubuntu LTS)
… or is setting up docker so simple that someone with Unix experience can get that done in no time?
It is really a huge time-killer if tests break on test-systems, but pass locally which is what this approach should hopefully avoid.
I use Docker a lot (maybe too much), and for Stan I have a Dockerfile like this
RUN apt-get update && apt-get install -y clang-3.9
RUN curl -OL https://github.com/stan-dev/cmdstan/releases/download/v$CSVER/cmdstan-$CSVER.tar.gz \
&& tar xzf cmdstan-$CSVER.tar.gz \
&& rm -rf cmdstan-$CSVER.tar.gz \
&& mv cmdstan-$CSVER cmdstan
&& cd cmdstan \
&& make -j8 build examples/bernoulli/Bernoulli
docker build -t cmdstan:2.17.1 .
then run w/ something like
alias runstan='docker run --rm -it -v `pwd`:/opt/cmdstan/work cmdstan:2.17.1 '
runstan make work/model
runstan work/model sample ...
runstan bin/stansummary ...
Similar images could be done for RStan, PyStan, etc and pushed to Docker hub so that Docker-savy users can just
docker pull stan/cmdstan
docker run stan/cmdstan ....
Hope that helps…
edit these work great with CI systems, including Travis whose packages are hopelessly out of date
edit fix wrong command
Thanks much. That should get me started.
Software served in a clean container sounds like a real solution.
ENV CXX flag import the relevant compiler into the docker image? If not, I wouldn’t think it would solve the containment problem.
cmdstan image should bring in an encapsulated version of Stan (with
math included with dependent libraries for Eigen and Boost).
Relevant compiler needs to be installed to a container. Then cxx flag works.
Not sure what does the “import” mean here.
ENV CXX sets a environment variable, which is picked up by Make. The compiler installed in the subsequent step
apt-get install ... clang-3.9.
Docker images like this can be based on existing images, so if there was a another base Stan image
stan/base, one would start the CmdStan Dockerfile with
FROM stan/base instead of the Python image I used above.
edit any toolchain configuration would be done once in the base image and then descendant images for eg CmdStan can assume a working toolchain.