CI alternatives

Developers
1

Here is a summary I found online comparing Drone, Travis, and Jenkins, all of which were brought up at the meeting today: https://stackshare.io/stackups/drone-io-vs-jenkins-vs-travis-ci

Having just spent a couple of months updating Jenkins to the new Pipelines that supposedly solve all problems, I’m not super keen on switching to something as untested and relatively unused in production as Drone is, unless anyone knows someone who uses it?

Travis might be interesting to try self-hosted but Travis has been pretty flaky for us even just doing our simplest and least taxing tests and I find it more difficult to debug when something does go wrong, but maybe others have different feelings about the differences?

Tagging @ariddell @syclik @Bob_Carpenter @mitzimorris @bgoodri @wds15

Daniel said:

I have no opinion about drone vs Jenkins. I just want something that works! That switch to pipelines seemed like it was a lot more effort than anticipated. And it doesn’t seem like it solved all the issues at hand. Anyway, this is the main reason I hesitate to replace things that work unless I see it working. It’s way too easy to claim it works on paper and it’s a lot easier to verify that it does or doesn’t once it’s done.

And I want to +1 that very heavily.

1 Like
2

From a friend at a large tech company:

ugh, travis. it’s very pretty. it’s not very good at actually running builds consistently. We also hit up against many race conditions in it that resulted in projects, builds, or people getting into stuck states. We use open source travis for our OSS projects, and it’s fine and our volume is low. We used hosted travis internally for our simpler projects (mostly simple library stuff) and ran into problems. It wasn’t workable for our more complex projects at all

1 Like
3

There’s also CircleCI, which scikit-learn uses. You can see it in action by looking at any arbitrary PR request, such as https://github.com/scikit-learn/scikit-learn/pull/11445 .

Looks like numpy uses CircleCI too. Pytorch uses jenkins.

4

I don’t think CircleCI has any C++ stories… plus a single 16 core test VM costs $9000 a year, which is $2k more than the $7000 we just paid for a 32 core / 64 thread machine that should last us a few years at least.