Please adjust ISO history handling (git hook?)
From our release process documentation:
ISO history ----------- Push the released ISO and USB images and their artifacts (`.buildlog`, `.build-manifest`, and `.packages` files) to our Tails ISO history git-annex repo, so that: - The Jenkins `build_IUKs` can fetch them. - Our isotesters can fetch them from there for their testing How to do so is described in the `ISO_history.mdwn` document in the RM team's Git repo. Then, wait (a few minutes) until the images appear on <https://iso-history.tails.boum.org/>.
It seems it can take up to 10 minutes before the new version appears, once the git annex copy is finished (which can take a while already, see #17414).
It would be great if this could be sped up, possibly by triggering the needed work in a git hook of some sort?
Thanks for your help.
#2 Updated by intrigeri about 1 month ago
- Category set to Infrastructure
- Status changed from New to Confirmed
The code that pulls from puppet-git.lizard is
tails::git_annex::mirror, deployed on www.lizard. It is currently a cronjob, running every 15 minutes.
I fully agree that triggering the needed work automatically, after the RM has uploaded the images to
gitolite3\@puppet-git.lizard:isos.git with git annex, would be ideal. It would also save resources on puppet-git.lizard, which tends to be a bit overloaded these days by all the Git clients hitting large repos repeatedly.
I don't remember how the gitolite / git-annex integration is done, but I hope it leaves us the possibility to add the hook we need here.
This design should be quite robust (e.g. concurrency is managed automatically), allows sysadmins to trivially start things by hand in the exact same context as when they happen automatically (as opposed to cron), and opens a very small attack surface:
- The Git server is allowed to ssh into a non-privileged user on the web server and it's only allowed to
toucha flag file in there.
- Pushing to the git annex repo triggers a hook that does the
ssh www touch $FLAGFILEdance.
- The web server has a
systemd.path(5)unit that monitors said file and triggers the
systemd.service(5)that does the pull. We can drop our current manual lock file handling as systemd does it by itself already.
- The cronjob can go away.
- If things get out-of-sync (e.g. a reboot or networking problem at the exact wrong time), the RM can trigger another sync by pushing a dummy change to the git annex repo.
If whoever has time first to implement this thinks it's over-engineered or preferred another design, by all means, be my guest.