Project

General

Profile

Feature #6216

Feature #6196: Build all active branches

Email notification on failed Jenkins build

Added by intrigeri about 6 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Continuous Integration
Target version:
Start date:
08/07/2013
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:
Starter:
No
Affected tool:

Related issues

Related to Tails - Bug #9660: Jenkins notifications doesn't cut our needs Resolved 06/30/2015

History

#1 Updated by Dr_Whax over 5 years ago

It seems that a plugin is needed in order to achieve this. A quick search pointed me to "email-ext"[1].

Email-ext about itself:

This plugin extends Jenkins built in email notification functionality by giving you more control. It provides customization of 3 areas.

Triggers - Select the conditions that should cause an email notification to be sent.
Content - Specify the content of each triggered email's subject and body.
Recipients - Specify who should receive an email when it is triggered.

[1] https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

Is this something we consider handy or is more information needed?

#2 Updated by intrigeri over 5 years ago

I would use whatever Holger picked for jenkins.debian.net: the INSTALL file in http://anonscm.debian.org/gitweb/?p=qa/jenkins.debian.net.git;a=summary says "The Jenkins EMail notification plugin is used". Is it the same? If unsure, ask :)

#3 Updated by Dr_Whax over 5 years ago

I asked Holger. He uses this plugin, which is a different one I previously listed: http://wiki.jenkins-ci.org/display/JENKINS/Mailer

He uses this plugin to also send irc notifications from the emails, apparently, it works nicer than kgb-bot. More scripts can be seen at: git://git.debian.org/git/qa/jenkins.debian.net.git

see bin/email2irc.sh
see bin/email2irc.sh and ./procmailrc

#4 Updated by intrigeri almost 5 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

tails-dev@ is now notified on failed ISO builds, and -l10n@ notified when the PO check on the master branch fails. Let's see how it goes.

#5 Updated by BitingBird over 4 years ago

I think we could mark this as resolved ?

#6 Updated by intrigeri over 4 years ago

  • Blocked by Feature #8482: Survey usual committers about dropping the collective pseudonym for signing commits added

#7 Updated by intrigeri over 4 years ago

I think we could mark this as resolved ?

No, the current setup is only a preliminary experiment. Now:

  • let's wait a month or two, and see if tails-dev@ subscribers complain;
  • most likely, we'll want more fine-grained notifications at some point (e.g. reporting to the person who broke stuff), as discussed on tails-dev@.

#8 Updated by intrigeri over 4 years ago

  • Blocked by deleted (Feature #8482: Survey usual committers about dropping the collective pseudonym for signing commits)

#9 Updated by intrigeri over 4 years ago

  • Assignee set to bertagaz
  • Target version set to Tails_1.4.1
  • Parent task set to #6196

Re-parenting so that the info here is easily available to bertagaz when he works on #6196.

#10 Updated by intrigeri over 4 years ago

https://gitweb.torproject.org/project/jenkins/jobs.git/tree/jobs.yaml might help: it has a quite extensive config for the email-ext plugin.

#11 Updated by bertagaz over 4 years ago

I'm not sure there really is a problem here. We're not using the email-ext plugin, but the Mailer plugin (used also on jenkins.debian.net) shipped with the Debian package.

We also hardcode the email that should be notified in our jobs configuration, so for automatic builds the notification on failure should be sent to the right recipient.

I'm not sure for the build triggered through the Jenkins web interface though.

We'll see once the automatic build feature is deployed I think, this needs some live testing.

#12 Updated by intrigeri over 4 years ago

I'm not sure there really is a problem here. [...] We also hardcode the email that should be notified in our jobs configuration, so for automatic builds the notification on failure should be sent to the right recipient.

That's thanks to this code snippet, right?

        elif branch['job_recipients'] == 'whoever_broke_the_build':
            self.job['publishers'].extend([
                {'email': {'recipients': '', 'sent-to-individuals': True}}
            ])

Was this tested and confirmed to do the right thing (as specified in the blueprint)?
(If not, please do test it, before we start spamming tons of people, in a way that should email you only if it works fine :)

#13 Updated by bertagaz over 4 years ago

intrigeri wrote:

I'm not sure there really is a problem here. [...] We also hardcode the email that should be notified in our jobs configuration, so for automatic builds the notification on failure should be sent to the right recipient.

That's thanks to this code snippet, right?

Yes.

[...]

Was this tested and confirmed to do the right thing (as specified in the blueprint)?
(If not, please do test it, before we start spamming tons of people, in a way that should email you only if it works fine :)

Tested in build 148 of the build_Tails_ISO_feature-8415-overlayfs job.
As you can see in the logs after setting the job accordingly and introducing a build failure, I confirm I did get the notification in my mailbox.

#14 Updated by intrigeri over 4 years ago

  • % Done changed from 10 to 50
  • QA Check set to Dev Needed

OK, great! Perhaps enable those settings on all jobs now, so that it is tested a bit more before we deploy #6196? Once this change is done, feel free to call this ticket resolved.

#15 Updated by bertagaz over 4 years ago

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

I've not enabled it for all jobs, I've updated the jobs config to reflect what would our setup be. Does it sound relevant?

#16 Updated by intrigeri over 4 years ago

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Ready for QA to Info Needed

I've not enabled it for all jobs, I've updated the jobs config to reflect what would our setup be. Does it sound relevant?

I'm sorry, I don't understand what you mean. I see no send-to-individuals anywhere in our jenkins-jobs repo. May you please rephrase what exactly you did, and what I should check?

#17 Updated by bertagaz over 4 years ago

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Info Needed to Ready for QA

Oops, my bad, forgot to push it. Done now, please have a look.

#18 Updated by intrigeri over 4 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 50 to 100

Done now, please have a look.

Looks good. Added commit 1502146 on top to fix a mistake in there, please have a look.
Calling this done, we'll file dedicated and more focused tickets if things go wrong.

#19 Updated by intrigeri over 4 years ago

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#20 Updated by intrigeri about 4 years ago

  • Related to Bug #9660: Jenkins notifications doesn't cut our needs added

Also available in: Atom PDF