Project

General

Profile

Feature #6439

Feature #6090: Automated builds

Fix cleanup of nightly built ISO images

Added by intrigeri almost 6 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Continuous Integration
Target version:
Start date:
11/25/2013
Due date:
07/15/2015
% Done:

100%

Feature Branch:
puppet-tails:bugfix/6439-fix-cleanup-of-nightly-built-ISO-images
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:

Description

http://nightly.tails.boum.org/build_Tails_ISO_experimental/ and http://nightly.tails.boum.org/build_Tails_ISO_devel/ only have artifacts for the last few days. In case it matters, these branches didn't build for a few days, so perhaps the script expects more files and was a little lost?


Related issues

Blocks Tails - Feature #6196: Build all active branches Resolved 08/07/2013 07/15/2015

History

#1 Updated by sajolida about 5 years ago

  • Assignee deleted (bertagaz)

#2 Updated by hyas about 5 years ago

Non-blocking ticket for 2.0

#3 Updated by intrigeri about 5 years ago

#4 Updated by intrigeri almost 5 years ago

I suspect the correct algorithm would:

  • keep one ISO for each of the last 8 days => we always have the most recent ISO from last week
  • keep one ISO per week over the last 5 weeks => we always have an ISO from ~ last month

But anyway, I'm pretty sure this is a classical problem, that has well-known good solutions.

#5 Updated by intrigeri over 4 years ago

  • Assignee set to Dr_Whax
  • Target version changed from Sustainability_M1 to Tails_1.4

#7 Updated by intrigeri over 4 years ago

  • Target version changed from Tails_1.4 to Tails_1.4.1

Postponing. Note that we can't postpone it once more, so if you doubt you can do it in time, please say so ASAP so we can reallocate this work without going crazy at the last minute.

The code lives as files/jenkins/slave/iso_builder/clean_old_jenkins_artifacts.rb in the tails Puppet module. Any of Python, Ruby or Modern Perl are acceptable languages, as long as the code is good and the dependencies in Wheezy + backports (we've not migrated our builders to Jessie yet).

#8 Updated by intrigeri over 4 years ago

  • Assignee changed from Dr_Whax to intrigeri

#10 Updated by intrigeri over 4 years ago

  • Priority changed from Normal to Elevated

#11 Updated by intrigeri over 4 years ago

For the record, a potential new contributor might want to take this one. If this doesn't happen by next Wednesday, I'll try hard to take care of it.

#12 Updated by intrigeri over 4 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to bertagaz
  • % Done changed from 0 to 50
  • Feature Branch set to puppet-tails:bugfix/6439-fix-cleanup-of-nightly-built-ISO-images

I've pushed something that seems to work on a local test data set. The algorithm may not be perfect yet, and I didn't bother refactoring the whole thing, but it seems to be better (given how the current algorithm deletes most ISOs we would like to keep, it can't be much worse anyway). Please review, and if happy merge and deploy!

#13 Updated by bertagaz about 4 years ago

  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 50 to 80

I've tested it on a local dataset too. It seems to work well, and have a better retention strategy than the previous version for sure, congrats!

So I've merged, deployed it, and tested it live. So far the build goes fine, and it doesn't seem to mess up with the artifacts.

If you believe this does close this ticket, please do. I'm not sure if we should wait some time now that the code is live to see how it behaves, even if I'm confident it works better.

#14 Updated by intrigeri about 4 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 80 to 100

So I've merged, deployed it, and tested it live. So far the build goes fine, and it doesn't seem to mess up with the artifacts.

Yay!

If you believe this does close this ticket, please do.

Done.

I'm not sure if we should wait some time now that the code is live to see how it behaves, even if I'm confident it works better.

We can still reopen this ticket (or create a new, more specific, one) if needed.

#15 Updated by intrigeri about 4 years ago

  • Assignee deleted (intrigeri)

#16 Updated by sajolida about 4 years ago

  • Due date set to 07/15/2015

Also available in: Atom PDF