Feature #5288: Run the test suite automatically on autobuilt ISOs
Implement the specified notification system for test suite failures on Jenkins
This is what follow from #10296 once we deem it's time to go ahead.
#6 Updated by bertagaz over 3 years ago
FYI this will be achieved by reverting commit
6dc5489 from the puppet-tails repo.
Then the test jobs in our JJB repo will still have the email notification activated, but the generation script will output only one email to be notified for the build and test jobs, which will be the same that we set up for the build jobs (
tails-rm or last commiter).
#9 Updated by intrigeri over 3 years ago
IMO this ticket should be marked as blocked by... whatever blocks it. #10296 is useful in itself, but it's simply too vague to express what we have to do before we can turn on the big red switch. I guess most of the blockers are subtasks of #10288. Once they're marked as blocking this ticket, then it'll be easier to sort them apart of the mass of #10288 subtasks, which will help a lot anonym and kytv organize their work :)
#10 Updated by intrigeri over 3 years ago
IMO this ticket should be marked as blocked by... whatever blocks it. [...] I guess most of the blockers are subtasks of #10288.
For what it's worth, I've had a quick look and I'm not so sure: the concerning amount of jobs that are aborted due to inactivity seems to be the top reason of non-successful test suite runs (out of the last 10 runs of
test_Tails_ISO_devel, 6 aborted that way, 2 were succesful, and 2 failed; on stable it's "only" 3 aborted jobs out of 10).
I don't know if email notifications go out for aborted jobs, so I don't know if/how this bug is blocking this very ticket, or rather #5288. bertagaz?
In any case, as requested a month ago (#9157#note-18): the aborted jobs bug seems to be very relevant to making #5288 deliver what it promised, so it's a pretty serious one, please track it with a ticket; its exact initial priority/timeline is a factor of the above question.
#16 Updated by bertagaz over 3 years ago
You'll see that there was a tiny bug like in test_Tails_ISO_web-10400-farsi-only #4 :
06:50:16 Sending e-mails to: $NOTIFY_TO 06:50:16 ERROR: Invalid Addresses
I've fixed that in commit
7170700 of the puppet-tails repo.
I've started a new build to test it: https://jenkins.tails.boum.org/job/build_Tails_ISO_web-10400-farsi-only/6/