New Jenkins Jobs Tutorial

maintenance

#1

Hey all,

I’ve been switching the Jenkins jobs over to pipelines. There are a few things that are changing now as a result from a user interface perspective.

Repo build status

You can now see a repo’s build status split out among branches and PRs:

and

Please ignore the “downstream tests” job status - this is the job that is triggered by downstream builds that want to test that e.g. a commit to the math repo works upstream with both CmdStan and Stan.

Job stages

Job stages are now shown in the new pipeline UI like so:


(You get to this page by clicking on the develop from the previous page)
You can see the logs here by clicking on a stage:

For debugging I also really like the pipeline steps view:


In this view you can click on the individual steps for just their console output:

For parallel build sections (many of them are now parallelized), the UI is pretty bad. So you will see all of the sections that were executed in parallel as having failed simultaneously in any of the graphical UIs. If the high level Console Output doesn’t give you any clues, the only way I’ve found to diagnose which stage actually failed is to click on each of them in the Pipeline Steps view and look for the one that doesn’t have the console output “Sending interrupt signal to process”.

The general job’s console output sometimes also has additional error messages, found here:
image

Retrying a job

Before, we would try to figure out the correct incantation to ask Jenkins to retest from the comments. While convenient if one remembered the correct incantation, often you had to click through and make sure it triggered anyway. Now the only way to retrigger a job is to go to the job page and click Build:
image
This will bring up a screen where you can input parameters. You only need to adjust these if you’re trying to test a specific branch combination across repos; otherwise leave them as their default values.
image
(I’ll add more text on this screen soon).

This will trigger the build and automatically update the PR’s status on github.

If there are any questions please ask them here! Thanks everyone.


#2

I think we should be thaking you. So, thank you. You don’t realize how many moving pieces there are until they get linearized as images!


#3

this is great - helpful hint - remember to log in to Jenkins


#4

I’m gonna sticky this thread for a while on the Developers category. Looks like we have some issues with the pipelines for Math right now.


#5

#6

Also, if there are issues where Jenkins isn’t running tests or they’re failing spuriously (or you think Jenkins is somehow broken), please post an issue to github for the repo involved, assign me and the “continuous integration” label if it’s not already posted. Thanks!