Project

General

Profile

Bug #16171

Feature #15292: Distribute a USB image

Bug #15999: Integrate USB image in the release process

Get our master branch ready for IDF v2 and new Tails Verification extension

Added by intrigeri about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
11/29/2018
Due date:
% Done:

100%

Feature Branch:
doc/16171-generate-idf-v2
Type of work:
Contributors documentation
Blueprint:
Starter:
Affected tool:

Related issues

Related to Tails - Bug #15998: Update design doc for Tails Verification Resolved 09/28/2018
Blocks Tails - Bug #15997: Test and release new Tails Verification Resolved 09/28/2018

Associated revisions

Revision cf383846 (diff)
Added by intrigeri about 1 year ago

Release process: adapt to IDF v2 (refs: #16171)

For now, we keep generating IDF v1 as well, the idf-generation script still
allows the case when we have only an ISO, and the release process assumes
there's an ISO but no USB image. Updating all this will be done in
a separate branch based on this one.

Revision 6747e63c (diff)
Added by intrigeri about 1 year ago

Add the IDF v2 for Tails 3.10.1 (refs: #16171).

Revision 789c1440 (diff)
Added by CyrilBrulebois about 1 year ago

Sort keys in dumped JSON output for IDF v2 (refs: #16171).

Revision c15c5007 (diff)
Added by intrigeri about 1 year ago

Make "git add" command work in all cases (refs: #16171)

v{1,2}/Tails/amd64/stable/latest.{yml,json} would fail because neither
v1/Tails/amd64/stable/latest.json nor v2/Tails/amd64/stable/latest.yml exist and
curly brackets generate strings mechanically, without taking into account what's
on the filesystem.

Thanks to kibi for spotting this problem and suggesting this solution.

Revision e7bbfdcb
Added by CyrilBrulebois about 1 year ago

Merge branch 'doc/16171-generate-idf-v2'

Fix-committed: #16171

History

#1 Updated by intrigeri about 1 year ago

  • Blocks Bug #15997: Test and release new Tails Verification added

#2 Updated by intrigeri about 1 year ago

  • Assignee changed from intrigeri to CyrilBrulebois
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA
  • Feature Branch set to doc/16171-generate-idf-v2

Hi kibi! This needs to be reviewed and merged in time before the 3.11 process starts (which is just a few days before our deadline for this project anyway).

#3 Updated by intrigeri about 1 year ago

Forgot to say: once happy, please merge into master → stable → devel.

#4 Updated by Anonymous about 1 year ago

Hi kibi, this currently blocks some of my work. Could you please review and merge this asap? thanks!

#5 Updated by CyrilBrulebois about 1 year ago

  • Assignee changed from CyrilBrulebois to intrigeri
  • QA Check changed from Ready for QA to Dev Needed
If nitpicking is fine:
  • the git add call will complain, as evidenced below.
  • I'd be happy to see a sort_keys=True added to the json.dumps call, so that one can easily read/compare the generated files. Calling bin/idf-content on a local image gave me a different key order compared to the committed one, which makes reading/reviewing slightly harder than it should be.

If that helps, I can probably fix both and merge, so that we get the ball rolling.

Example of git add being unhappy:

$ mkdir 1 2
$ touch 1/latest.yml 2/latest.json
$ git add {1,2}/latest.{yml,json}
fatal: pathspec '1/latest.json' did not match any files

That's due to curly brackets generating strings mechanically, without taking into account what's on the filesystem, hence git add complaining.

#6 Updated by CyrilBrulebois about 1 year ago

Pushed trivial fixes (that can be squashed together) in the pu/doc/16171-generate-idf-v2 branch, I think the git add issue can be solved with a wildcard (latest.*) even if we lose some bits of information.

Another thing that might be nice to have is a more uniform “image description format”; that's not been done across the whole site at the moment:

$ git grep -l 'ISO description file'
wiki/src/contribute/build/reproducible.mdwn
wiki/src/contribute/design/verification_extension.mdwn
wiki/src/contribute/release_process.mdwn

Again, nothing critical.

In the meanwhile, I've merged and pushed the branch into master, as it seemed good enough to merge, and u needed it to get other things going.

I haven't merged into other branches yet, as it seems it would be slightly better to get the details mentioned above fixed (or dismissed as unimportant) before doing so. Feel free to correct me, still learning the ropes.

#7 Updated by Anonymous about 1 year ago

  • Assignee changed from intrigeri to CyrilBrulebois
  • QA Check changed from Dev Needed to Ready for QA

Thank you! I've been able to mostly complete my work on #15995 :)

#8 Updated by intrigeri about 1 year ago

Pushed trivial fixes (that can be squashed together) in the pu/doc/16171-generate-idf-v2 branch

Great, thanks! Merged into master.

, I think the git add issue can be solved with a wildcard (latest.*) even if we lose some bits of information.

ACK, will do.

Another thing that might be nice to have is a more uniform “image description format”; that's not been done across the whole site at the moment:

Fully agreed but IMO that should be covered by #15998: given that work has not been done yet, new terminology is not set in stone nor applied consistently, so I had to guess and improvise a bit here. IIRC I've done some of this work on my branch for #15999 anyway. Keep in mind that this branch is a transitional one merely meant to ensure we generate IDF v2 during our next releases. The full blown thing is #15998 and #15999 :)

I haven't merged into other branches yet, as it seems it would be slightly better to get the details mentioned above fixed (or dismissed as unimportant) before doing so.

Well, merging master into the release branch is part of our release process and the RM is supposed to follow the version of the doc that's on the release branch, so:

  • Either the RM follows the version that's in stable before merging master, and then they won't generate IDF v2 and we have a serious problem.
  • Or the RM follows the version that'll be in stable after they merge master into it, and then they'll be affected by those small problems.

So I'm not sure it makes much sense to differentiate "merge this into master" from "merge this into stable". In any case, no big deal at all: I'm confident this work will reach the point when you're happy to merge it everywhere by the end of the week :)

#9 Updated by intrigeri about 1 year ago

, I think the git add issue can be solved with a wildcard (latest.*) even if we lose some bits of information.

ACK, will do.

Done, tested with your reproducer, works fine. Once happy, please merge into master → stable → devel.

#10 Updated by CyrilBrulebois about 1 year ago

  • Status changed from In Progress to 11
  • % Done changed from 50 to 100

#11 Updated by intrigeri about 1 year ago

  • Status changed from 11 to Resolved
  • Assignee deleted (CyrilBrulebois)
  • QA Check changed from Ready for QA to Pass

Thanks!

#12 Updated by Anonymous about 1 year ago

  • Related to Bug #15998: Update design doc for Tails Verification added

Also available in: Atom PDF