- All the tests are running, except for CmdStan; I’ve had some issues with 2.20 and the static crtp initialization. Since I was previously compiling multiple models into a fat unit test binary. I need to spend some time looking into this.
Distribution tests are handled by setting the appropriate variable at configure time, e.g.
cmake .. -G"Ninja" -DSTAN_BUILD_RVECTESTS=TRUE -DCMAKE_BUILD_TYPE=Release && ninja && ctest --output-on-failure -j<n_processors>. It should work, although I haven’t tested it in a few weeks, since I have to take special measures to run it due to how much disk space the artifacts take up.
- Half the failures are from the OpenCL tests failing due to an issue in 2.20 resolved here: https://github.com/stan-dev/math/issues/1297 - I just haven’t got around to rebasing again because I’m a bit wary of chasing ^HEAD until I’ve fixed everything else. If you look in the dashboard report, some of them may not be suffixed with
CL, but indeed they are part of the OpenCL unit testing. Probably I ought to add a suffix to all tests that are part of the OpenCL tests. It’ll be kind of ugly though since the GTest name for the test will be “Foo.BarClCl”; a lot of the OpenCl tests call out to base Stan Math tests in cholesky... etc.
The other half, I need to spend quality time with the codebase and debug. Pull requests welcome :).
- Directory structures in src: I’m a little alarmed that you’re surprised by the directory structures, since this arrangement is what I thought was desired by you and @Bob_Carpenter. Can you explain how it’s different from what you expected?
Directory structures in tests: I pretty much just copied and flattened the existing test directories in Stan/Math/CmdStan.
-
How the CMake builds are supposed to work:
Well, hopefully, as in the Quick Start I wrote in the readme, building should look like:
git clone <repo_url> && cd <repo_url>mkdir build && cd buildconan install .. --build=missingcmake .. && cmake --build . -j<n_procs>
I’ve created virtual targets for the different configurations of Stan Math (Stan, Stan-with-opencl, Stan-with-mpi, Stan-with-multithreading) and you pick and choose the one you want in the build file if you’re a downstream user using CMake. Otherwise if you’re not you manually set all the header options as you did before.
As an aside, this means I generate different stanc for base Stan and stanc-with-mpi since there are MPI dependent ifdefs in the source.
All this should be pretty seamless for users since I’ve written build logic to generate these virtual targets if and only if the appropriate dependencies have been found in the system.
This also means the CI builder doesn’t really need to know anything about configuration - if the dependencies are present, the binaries/tests are automatically compiled and registered with the CTest test runner. If they aren’t, nothing happens.
- Using a package manager
So I guess one big difference is that I’ve included a call to conan in the quick start build steps. My goal while doing this was to make it possible to build Stan without using hard-coded dependencies in the build tree. Instead it’s managed via CMake find_package calls. Then whoever is using Stan and packaging it can seamlessly require the use of their dependencies without modifying the build script - all they have to do is make sure their path is set correctly. For example, when I am compiling in MSYS2, I don’t even need Conan since I have Sundials installed in /usr, and everything else has a system package. Still, we want to support one-command build so calling out to a cross-platform package manager seemed to be the easiest way to support that.
Also, Conan as a package manager platform allows binary artifacts to be installed side-by-side as build-requirements, so you can do stuff like distribute stanc3 binaries or Eigen-with-BLAS support in one command. I haven’t documented how to do this but on my Linux environment I use it to install CMake into a virtual environment since the system CMake is sort of ancient.
- Packing things into fat test binaries.
One other change is that instead of building individual executables for each source file, I compile them into .obj and link them together with a single GTest main.
This is mostly for developer ease of use. The CMake ecosystem is not designed to handle mass amounts of small binaries. Each target gets an individual entry in the CMake build graph. Each time a build is triggered CMake traverses the entire graph in order to check for changes in dependencies. This is good when the number of build targets is small compared to the number of sources. CMake automatically only rebuilds dependencies that change. When testing small changes in the code base, CMake configuration time can dominate compilation and link time, even in Stan is which full of large, repetitive template instantiations. It also makes the project must more friendly to IDE users who must browse through all targets individually in tiny dropdown menus. Basically a compile target is much more heavyweight in CMake than a Makefile target; each one individually has or has the capability of having its own compile flags, linker flags, etc.
If you wish to run an individual test, CTest supports filtering tests by name, or by label (although I haven’t used that feature yet in the codebase, it exists). There’s also running tests by binary - in Stan Math, in order to cut down linking times I’ve separated out the unit tests into smaller groupings (see CMakeLists for details). I’m not against multiple binaries, just that if there’s N sources log(N) test binaries is the way to go. I realize that one big binary has its own issues with link time. I recommend everybody working on Stan use the LLD linker, Ninja, and ccache while doing incremental building and testing.