Build tool - make vs?




I think I remember @sakrejda mentioning in a Stan meeting that he was working on a cmake version of Stan (not sure which repo this was, maybe math?). Is that still going on? I’ve been wondering how difficult it would be to switch to a more modern system in the hopes of speeding builds up and doing incremental builds safely more often (esp. for testing). I’ve heard Bazel highly recommended by people, though they typically have had some Google affiliation in their past. I have no personal experience with any of the C++ options outside of make so curious what others think.



The issue here is user-friendliness – the current make system has been specialized sufficiently well that it works out of the box for most systems we encounter, but we’ve had no end of trouble with tools like cmake. If we can find something that works better than make, works out of the box on Linux, Mac, and Windows, and we’re willing to translate the current build configuration over then I think most of us would be receptive.


I did get cmake working in a few different contexts but once I was done with it it turns out I needed some command-line invocations in there to get the protocol buffers and Stan models build. The error messages are terrible (it’s an auto-generated make-file so… :/ )

I will give Bazel a shot at some point but I tried make/cmake/R’s remake and one of the Python-based tools and all of them have weird stuff embedded. For straight C++ I’d go with Bazel or cmake but Stan isn’t straight c++ (we sometimes auto-generate c++). Maybe going by the Bazel protobuf scripts we could do it right in that system but I haven’t had time to push it.



Others have also tried to port our makefiles to cmake and eventually gave up.

My own experience installing third party software is that cmake almost always fails in cryptic ways. Whereas makefiles usually work for me.

I’m pretty sure the bottleneck in our testing is because of dependencies, not because make is slow or bad at figuring them out.

Because the tests bring in high-level headers (like prim/mat.hpp), any changes to any included file in mat.hpp requires us to rebuild the world. I’d be surprised (but very happy!) if another system could get around this in a cleverer update only if the things you use in a header change, not if the header itself changes.

Another reason things are so slow is that Stan’s largely header only and the template metaprograms are very slow to compile.

A third reason is the combinatorics in the probability functions. We’re not comfortable with fewer tests because the underlying code can branch on any of the distinctions in input type to have different behavior. They’ve saved us on many occasions.


The appealing feature of cmake is that you can specify c++ builds really cleanly but the moment you have to do something more complex (e.g.-our testing) the scripts turn into a nightmare of custom_command/target thrown all over the place.


I’d be happy if we could move from make. I’ve thought about this for a while. Here’s what we need to do:

  • Math. For developers only:

    • Build CVODES library
    • Build GTEST
    • Be able to generate tests
    • Build individual unit tests
    • Build doxygen doc
  • Stan. For developers only:

    • Build CVODES library
    • Build GTEST
    • Build a test version of the Stan compiler
    • Build individual unit tests
    • Build doxygen doc
    • Build latex manual
  • CmdStan. For users:

    • Build the Stan compiler
    • Build stansummary
    • Build CVODES library
    • Pass Stan program through the compiler, then compile the generated C++
  • CmdStan. For developers:

    • All of the above
    • Build doxygen documentation
    • Build manual

It’s easy to require developers to install more tools. For the users, we should think about dependencies. The way we currently have our CmdStan build system, the dependencies are:

  • Linux: preinstalled make, C++ compiler
  • Mac: XCode installed make, C++ compiler
  • Windows: RTools installed make, C++ compiler


Why do we use make during runtime / for users? Can we keep that subsystem the same while upgrading the developer build system?


What do you mean "make during runtime / for users"?


Oh, I think maybe I misinterpreted what you were saying - for the CmdStan users you meant users who are compiling CmdStan, not users who are using CmdStan. Are there a lot of people who do that?


To use cmdstan you need make.


CmdStan users need to have make to run CmdStan. If they have a Stan program, say foo.stan, they need to be able to create an executable from foo.stan. This requires:

  • bin/stanc having been built
  • Passing foo.stan to bin/stanc generating foo.cpp
  • Compiling foo.cpp to an executable

We don’t strictly need make to compile files.


Ah, right, sorry. So using CmdStan implies compiling the rest of Stan and the math library? Hm, yeah it would suck a bit to force users to install something like Bazel to use CmdStan…


In broad strokes, yes, that’s what it implies. We could add a shell script and a Windows batch script to compile if we even wanted less dependencies. But I find that make is typically installed with whatever toolchain is available.

Most of Stan is still header-only, so we just need to include Stan and Math libraries. CVODES is the exception, and in fact, we can compile without the library if necessary (and if we don’t use the CVODE ODE solver). We tend to just build it because we’ve figured out that process reliably.