Project

General

Profile

Feature #16059

Improve UX for scheduling builds for newly pushed branches on the CI

Added by segfault 9 months ago. Updated 8 months ago.

Status:
Confirmed
Priority:
Normal
Assignee:
-
Category:
Continuous Integration
Target version:
-
Start date:
10/16/2018
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Currently, builds have to be scheduled manually, once the jobs appear on Jenkins. The cronjob to update the jobs runs every 5 min, and the jobs update takes 1-2 min. The resulting worflow is:

1. Push branch with changes you want to test
2. Wait 2-7 minutes until the corresponding job appears on Jenkins
3. Manually schedule a build for the job on Jenkins


Related issues

Related to Tails - Feature #9741: New jobs in Jenkins should be built immediately Rejected 07/14/2015

History

#1 Updated by intrigeri 9 months ago

  • Category set to Continuous Integration

#2 Updated by intrigeri 9 months ago

  • Subject changed from Improve UX for scheduling builds on the CI to Improve UX for scheduling builds for newly pushed branches on the CI

#3 Updated by intrigeri 8 months ago

  • Related to Feature #9741: New jobs in Jenkins should be built immediately added

#4 Updated by intrigeri 8 months ago

Note that this idea surfaced in #9741 a while ago and was rejected back then. I think that was for wrong reasons: my arguments that lead bertagaz to close the ticket are both mistaken.

#5 Updated by bertagaz 8 months ago

I agree it's a bit of a suboptimal UX for developers. Now I've think a bit about it in the past, and the conclusions were mitigated.

First, it's not that easy to implement. That's the way Jenkins behaves natively. There might be plugins to workaround this, but I doubt it.

So to have that behaviour, we'd have to hack something in our jenkins job creation scripts. That would not be in the jenkin-job-builder part, I don't think there's a way to do that here. So that would be in the pythonlib code, where we parse the Tails Git to get the new and merged branches to generate the job list. At that time, we could probably make a HTTP request to the URL that trigger a build, except the job does not exist yet. Bummer. So that would be another piece of code, after the JJB run, that find somewhere the information about which jobs are new, in order to do the HTTP requests to trigger the builds. Maybe by running again the pythonlib code, which takes some times.

OTOH, in the past I've been aware of this, so when I pushed a new branch, I knew if I wanted to have at that point feedbacks from Jenkins I had to push a button in the web interface. Or before working, I forked the new branch, pushed it so that itsjob gets created in Jenkins, and then hacked on it and pushed the new commits, so they get tested right away.

So all in all, I've been wondering if this little UX problem was worth the development time and added code and infra complexity. Once a developer knows that, she also knows how to workaround it. And now that more Tails people have access to the Jenkins GUI, it leverages a bit more the need for that code. Maybe some documentation would be enough?

I also think that your second argument in #9741#note-2 is reasonable.

#6 Updated by intrigeri 8 months ago

Thanks for the input!

So that would be another piece of code, after the JJB run, that find somewhere the information about which jobs are new, in order to do the HTTP requests to trigger the builds.

Yep: a Git post-receive hook in jenkins-jobs.git would have access to all the info it needs to tell which jobs were created and send the corresponding HTTP requests. This may require that we move our post-update code to post-receive too so we have guaranteed ordering (apparently, whether post-receive executes before or after post-update is undefined behaviour, but I did not research this much).

Or before working, I forked the new branch, pushed it so that itsjob gets created in Jenkins, and then hacked on it and pushed the new commits, so they get tested right away.

Interestingly, this was my exact argument last time we had this conversation, and you nicely explained why this can't possibly work: if the new branch has no commit on top of its base branch, our automation won't create a job for it yet.

So all in all, I've been wondering if this little UX problem was worth the development time and added code and infra complexity.

On the one hand, the solution described above seems pretty cheap in terms of all these cost metrics. Also, I suspect the UX cost of every such little papercut adds up in a non-linear fashion because one resents a UX bug differently, depending on whether it's isolated or the third one in a row they have to ask me on XMPP why the system won't do its job as expected. So I'm not sure if it makes sense to do UX cost/benefit analysis in a non-holistic way.

OTOH, now is probably not the best time to invest a lot into our current job generation setup:

  • If we stick to Jenkins: with Jenkins 2+ it seems there are good options to fully replace the way we generate jobs, in a way that Jenkins understands and can react sensibly to (as opposed to "hi Jenkins! things happened, we won't tell you what, but here's your new set of jobs, deal with it" that we're doing now and which causes this sort of problems).
  • We want to move to GitLab at least for merge requests and quite possibly for issues tracking. At this point it's not clear to me whether the cost of migrating the CI too will be greater than the benefits it would bring.

So well, for now, IMO let's keep this open in case one of us feels like doing some structured procrastination on a rainy day, but unassigned and not urgent.

Maybe some documentation would be enough?

This would be great!

Also available in: Atom PDF