How to generate (or just recompile) only one distribution test

Is it possible to generate only one distribution test at a time? Often when debugging failures in those tests is enough to generate one and recompile it over and over while looking for a fix. Even just compiling a single file from the distribution tests (say test/prob/distr/distr_cdf_test.hpp), it generates tens of test files that need to be compiled and it can take forever.

It would be nice if there was a way to generate only one of the test .cpp files. Or, alternatively, I could live with generating all tests once, but then being able to recompile only one of the generated tests rather than starting again from the test header.

I tried with make distr_cdf_00000_generated_fd_test.cpp, make distr_cdf_00000_generated_fd_test and similarly using runTests.py instead of make, but they all failed.

Does anyone has tricks up their sleeves they could share?

Yes, it’s possible and straightforward to do. First off, apologies… the testing framework was expanded without additional documentation and clear instructions before we were really doing more careful code reviews. It’s some of the technical debt we have to pay down at some point.

From a completely clean Math repo, you can start by typing:

> make generate-tests

This generates all the hpp files all at once. If you wanted to generate just one of the distributions, you could do something like:

> test/prob/generate_tests test/prob/normal/normal_cdf_test.hpp 100

The test/prob/generate_tests is an executable that’s created that generates thesee tests. It takes two arguments. The first is the header file that is used to generate the individual tests and the second is the number of template instantiations within a single file. If that’s larger, there are less files and the compile times take longer per file. We have this because certain compilers are unable to instantiate everything all at once.

Once that’s generate, it looks like a normal unit test. You can run a single test like:

> ./runTests.py test/prob/normal/normal_cdf_00000_generated_v_test.cpp 

That will run that single test. The v vs fd vs fv etc indicates whether it’s reverse mode, forward mode, higher order instantiations in the file.

Hope that helps.

2 Likes

Thank you! This works and is exactly what I was looking for!

And as I went to the wiki page to add your explanation, I found that all information was already there and I must have missed it last time I read it! D’oh! I’ve added a bit from your message to complete the explanation.

Thanks for updating it! I’m glad I had written it down a while ago.

It is still technical debt that we’re struggling with. I believe it’s a lot easier to get something better in place with c++11.