First, the automatic performance test reports are great. Here are a few things which we could improve/I have questions:
- How to interpret the numbers… I mean is >1 better or <1? I am never sure.
- Could the posted report include a link to the actual Jenkins run? Or some reference which makes it easy to find the respective logs? I struggle with that a lot.
- Would it be possible to output a few more details like compiler variant, compiler version, os & CPU used?
- Would it also be possible to request performance tests to run on a specific platform (linux/macOS/Windows)?
The above is ordered in importance according to my taste, of course.