Project

General

Profile

Feature #15524

Feature #14468: Add VeraCrypt support to Tails

Feature #15214: Iteration 1: Support unlocking VeraCrypt partitions in GNOME

Iteration 1: Write release process documentation for custom packages

Added by segfault 7 months ago. Updated 28 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
04/11/2018
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
feature/15524-veracrypt-release-documentation
Type of work:
Contributors documentation
Blueprint:
Starter:
Affected tool:

Description

In the releases between when we ship our VeraCrypt work in Tails (3.9) and when Tails is based on Buster, we need to maintain the custom packages we create (see #15521).

For this we should include a step in the release process to check whether any of the packages we base ours on have an update, and in that case merge the update into our packages.


Related issues

Related to Tails - Feature #14728: Track security updates during the Tails code freeze Confirmed 09/26/2017

Associated revisions

Revision 79a140ca (diff)
Added by segfault about 2 months ago

VeraCrypt release docs: Fix some issues (refs: #15524)

Revision 1c5223fe (diff)
Added by segfault 30 days ago

VeraCrypt release docs: Add step to update package version (refs: #15524)

Revision e845ef1c (diff)
Added by segfault 30 days ago

VeraCrypt release docs: Fix incorrect command (refs: #15524)

Revision 5434f2df (diff)
Added by segfault 30 days ago

VeraCrypt release docs: Fix distribution name (refs: #15524)

Revision 80a024d4 (diff)
Added by segfault 30 days ago

VeraCrypt release docs: Quote variables (refs: #15524)

Revision 1766281b (diff)
Added by segfault 30 days ago

VeraCrypt release docs: Fail if variable unset (refs: #15524)

Revision 8e03989e (diff)
Added by intrigeri 29 days ago

Don't include shell quotes in APT sources (refs: #15524)

Revision 3c6224f5
Added by intrigeri 29 days ago

Merge branch 'feature/15524-veracrypt-release-documentation' (Closes: #15524)

History

#1 Updated by bertagaz 6 months ago

  • Target version changed from Tails_3.7 to Tails_3.8

#2 Updated by intrigeri 5 months ago

When working on this you can assume the RM knows the basics of Debian packaging so I think this can boil down to:

  • document the list of custom packages and their inter-build-dependencies as you did on #14481
  • something like "download the updated Debian source package, the old Debian source package that our custom package was forked off, and our custom source package, debdiff the 2 last ones, apply that debdiff on top of the 1st one, dch and boom"

So I don't think we need one dedicated page per package under wiki/src/contribute/release_process/, one single page referenced from contribute/release_process should be enough.

#3 Updated by intrigeri 5 months ago

  • Target version changed from Tails_3.8 to Tails_3.9

This can wait: as long as it's done (i.e. implemented, reviewed and merged) in time for 3.9 we're good.

#4 Updated by segfault 4 months ago

  • Assignee changed from segfault to intrigeri
  • QA Check set to Info Needed

intrigeri wrote:

  • something like "download the updated Debian source package, the old Debian source package that our custom package was forked off, and our custom source package, debdiff the 2 last ones, apply that debdiff on top of the 1st one, dch and boom"

How is the RM supposed to download the old Debian source package? It doesn't seem possible to download these via apt source (I tried to download the previous version of udisks2 on stable via apt source udisks2=2.1.7-3 and it doesn't work). Should we upload them somewhere?

#5 Updated by intrigeri 3 months ago

  • Assignee changed from intrigeri to segfault
  • QA Check deleted (Info Needed)

segfault wrote:

intrigeri wrote:

  • something like "download the updated Debian source package, the old Debian source package that our custom package was forked off, and our custom source package, debdiff the 2 last ones, apply that debdiff on top of the 1st one, dch and boom"

How is the RM supposed to download the old Debian source package?

With apt-get source or dget.

It doesn't seem possible to download these via apt source (I tried to download the previous version of udisks2 on stable via apt source udisks2=2.1.7-3 and it doesn't work). Should we upload them somewhere?

Debian publishes the source of all packages so we don't have to upload them ourselves. Stretch has 2.1.8-1 and I see no Debian release with 2.1.7-3 which I suspect is why what you've tried did not work.

#6 Updated by segfault 3 months ago

  • Assignee changed from segfault to intrigeri
  • QA Check set to Info Needed

Debian publishes the source of all packages so we don't have to upload them ourselves. Stretch has 2.1.8-1 and I see no Debian release with 2.1.7-3 which I suspect is why what you've tried did not work.

Yes, but 2.1.7-3 was once released, and we are talking about old Debian packages, i.e. ones that are not the current version. So what I'm looking for is a way to download those, which does not seem possible with apt source or dget.

#7 Updated by intrigeri 3 months ago

  • Assignee changed from intrigeri to segfault
  • QA Check changed from Info Needed to Dev Needed

we are talking about old Debian packages, i.e. ones that are not the current version.

Right, I missed that part of the context, sorry. Note that I think we only need that for packages that were not maintained in Git when Stretch was released (e.g. most GNOME stuff). For packages that had a Vcs-Git back then already, better fork that Vcs-Git and then updating the package is essentially a Git merge + gbp dch.

So what I'm looking for is a way to download those, which does not seem possible with apt source or dget.

https://snapshot.debian.org/

#8 Updated by intrigeri 2 months ago

  • Target version changed from Tails_3.9 to Tails_3.10.1

#9 Updated by segfault 2 months ago

What should the distribution field be set to in new versions of our custom packages?

#10 Updated by segfault 2 months ago

  • Assignee changed from segfault to intrigeri
  • QA Check changed from Dev Needed to Ready for QA
  • Feature Branch set to segfault:feature/15524-veracrypt-release-documentation

I wrote a draft. I didn't document the package uploading because I don't know how that is done.

#11 Updated by intrigeri 2 months ago

What should the distribution field be set to in new versions of our custom packages?

It should be based on the name of the branch (bin/add-APT-overlay does the translation for you) that will bring the update, which itself should be based on the corresponding ticket number.

#12 Updated by intrigeri 2 months ago

  • Assignee changed from intrigeri to segfault
  • QA Check changed from Ready for QA to Dev Needed

I wrote a draft.

Great!

  • I would not hard-code the "version in Tails 3.9" info. I don't think it's terribly useful and we would need additional steps to update it.
  • s/a new version in Debian stable/a new version in Debian stable or stretch-security/
  • Let's make the instructions independent from 3.9 (otherwise we would need additional steps to update them): instead of pulling our forked source package from the 3.9 suite, let's pull it from our stable, testing or devel suite, depending on which release the updated packages should go in. And then drop check-valid-until=no for these APT sources.
  • Why do we need deb (binary) APT sources? I'd rather not encourage developers to add those to their main system unless it's really needed.
  • "and the original version" feels ambiguous to me; right now I know what it means but in 6 months, well… Maybe "the version of the official Debian package it was forked off" or similar?
  • Please specify which kind of chroot one must build the package in. E.g. https://tails.boum.org/contribute/release_process/tails-greeter/ reads "(use a Stretch/amd64 chroot)".
  • It seems to me that the "Download the new source package" section assumes that one has no APT source configured with a newer udisks2 than the one in Debian stable and stretch-security. That's not always the case, o.g. on my system this would download the version from sid, which is not what we want.
  • s/>> tails.diff/> tails.diff/ otherwise things will break as soon as one repeats the process, be it to update the same package or another one.
  • I would "automate" the process a bit by first storing all the needed info in variables (source package name, version in current Tails stable/testing/devel, version we forked off last time, new version we want to rebase on top of, etc.) so that once it's done, the rest of the process is a matter of copy'n'paste without having to adjust every second command by hand. Not a blocker, YMMV :)
  • s/hit Debian stable/have hit Debian stable/
  • Ideally this would not be part of the release process: it feels a bit too late to act, there won't be any 3rd-party QA, and I don't think this should be on the RM's plate anyway. To me it looks much more like FT work. But we have no good way to track the need for such updates yet (#14728 will help) so unless you have a better idea, let's keep things as-is and add a note about it on #14728.

I won't bother testing the doc right now. Let's do it the first time we need to follow it :)

I didn't document the package uploading because I don't know how that is done.

You can copy the "Add the Debian package to Tails" section from https://tails.boum.org/contribute/release_process/tails-greeter/ :)

(Slightly off-topic: BTW, if you're curious, https://tails.boum.org/contribute/APT_repository/custom/ has more detailed info.)

#13 Updated by intrigeri 2 months ago

  • Related to Feature #14728: Track security updates during the Tails code freeze added

#14 Updated by segfault about 2 months ago

  • Assignee changed from segfault to intrigeri
  • QA Check changed from Dev Needed to Info Needed

intrigeri wrote:

I wrote a draft.

Great!

  • I would not hard-code the "version in Tails 3.9" info. I don't think it's terribly useful and we would need additional steps to update it.

I think it is useful. How would the workflow to check for a new version look like if the version is not listed in the instructions?

  • s/a new version in Debian stable/a new version in Debian stable or stretch-security/

Done.

  • Let's make the instructions independent from 3.9 (otherwise we would need additional steps to update them): instead of pulling our forked source package from the 3.9 suite, let's pull it from our stable, testing or devel suite, depending on which release the updated packages should go in.

Done.

  • Why do we need deb (binary) APT sources? I'd rather not encourage developers to add those to their main system unless it's really needed.

I somehow thought that those were also required to download the sources. I removed them.

  • "and the original version" feels ambiguous to me; right now I know what it means but in 6 months, well… Maybe "the version of the official Debian package it was forked off" or similar?

Done.

Done.

  • It seems to me that the "Download the new source package" section assumes that one has no APT source configured with a newer udisks2 than the one in Debian stable and stretch-security. That's not always the case, o.g. on my system this would download the version from sid, which is not what we want.

Fixed.

  • s/>> tails.diff/> tails.diff/ otherwise things will break as soon as one repeats the process, be it to update the same package or another one.

Fixed.

  • I would "automate" the process a bit by first storing all the needed info in variables (source package name, version in current Tails stable/testing/devel, version we forked off last time, new version we want to rebase on top of, etc.) so that once it's done, the rest of the process is a matter of copy'n'paste without having to adjust every second command by hand. Not a blocker, YMMV :)
  • s/hit Debian stable/have hit Debian stable/

Done.

  • Ideally this would not be part of the release process: it feels a bit too late to act, there won't be any 3rd-party QA, and I don't think this should be on the RM's plate anyway. To me it looks much more like FT work. But we have no good way to track the need for such updates yet (#14728 will help) so unless you have a better idea, let's keep things as-is and add a note about it on #14728.

OK, I don't have a better idea.

I won't bother testing the doc right now. Let's do it the first time we need to follow it :)

ACK.

I didn't document the package uploading because I don't know how that is done.

You can copy the "Add the Debian package to Tails" section from https://tails.boum.org/contribute/release_process/tails-greeter/ :)

Done.

(Slightly off-topic: BTW, if you're curious, https://tails.boum.org/contribute/APT_repository/custom/ has more detailed info.)

OK, thanks.

#15 Updated by intrigeri about 1 month ago

  • Assignee changed from intrigeri to segfault
  • QA Check changed from Info Needed to Dev Needed

Hi!

All your fixes look good, woohoo! The remaining fixes I'm requested below should be easy so I'm very confident I'll merge your branch after the next iteration of the feedback loop :)

  • I would not hard-code the "version in Tails 3.9" info. I don't think it's terribly useful and we would need additional steps to update it.

I think it is useful. How would the workflow to check for a new version look like if the version is not listed in the instructions?

Either look into the .packages file on our master branch, or use dpkg in a Tails system, or use APT to query the tagged APT snapshot for the latest Tails release.
Now, if you prefer to keep this duplicated info on this document, fine, but please add steps to update it.

  • Ideally this would not be part of the release process: it feels a bit too late to act, there won't be any 3rd-party QA, and I don't think this should be on the RM's plate anyway. To me it looks much more like FT work. But we have no good way to track the need for such updates yet (#14728 will help) so unless you have a better idea, let's keep things as-is and add a note about it on #14728.

OK, I don't have a better idea.

Fine, please update #14728 then :)

Also, I've noticed some regressions since the last version I've reviewed:

  • I believe apt source -t ${NEW_VERSION} ${PACKAGE_NAME} won't work. Maybe you meant apt source ${PACKAGE_NAME}=${NEW_VERSION}?
  • without "bugfix/" or "feature/" feels wrong.
  • All these unquoted variables hurt my eyes. Yeah, shell scripting is hard :/ Our preferred/standard way to write variables in such documents is ${VARIABLE:?} which is basically equivalent to set -u ; "$VARIABLE" i.e. an error will be raised if whoever pastes these command lines has not set $VARIABLE (mistakes do happen :)

#16 Updated by segfault 30 days ago

  • Assignee changed from segfault to intrigeri
  • QA Check changed from Dev Needed to Ready for QA
  • Feature Branch changed from segfault:feature/15524-veracrypt-release-documentation to feature/15524-veracrypt-release-documentation

intrigeri wrote:

Hi!

All your fixes look good, woohoo! The remaining fixes I'm requested below should be easy so I'm very confident I'll merge your branch after the next iteration of the feedback loop :)

  • I would not hard-code the "version in Tails 3.9" info. I don't think it's terribly useful and we would need additional steps to update it.

I think it is useful. How would the workflow to check for a new version look like if the version is not listed in the instructions?

Either look into the .packages file on our master branch, or use dpkg in a Tails system, or use APT to query the tagged APT snapshot for the latest Tails release.

All of these methods need manual effort. I think the process will be a lot faster if we provide the version to compare with in the document. And since we don't expect many updates, I expect the overhead of keeping this list up to date to be less than the time we save each time we check for updates.

Now, if you prefer to keep this duplicated info on this document, fine, but please add steps to update it.

Done.

  • Ideally this would not be part of the release process: it feels a bit too late to act, there won't be any 3rd-party QA, and I don't think this should be on the RM's plate anyway. To me it looks much more like FT work. But we have no good way to track the need for such updates yet (#14728 will help) so unless you have a better idea, let's keep things as-is and add a note about it on #14728.

OK, I don't have a better idea.

Fine, please update #14728 then :)

Done.

Also, I've noticed some regressions since the last version I've reviewed:

  • I believe apt source -t ${NEW_VERSION} ${PACKAGE_NAME} won't work. Maybe you meant apt source ${PACKAGE_NAME}=${NEW_VERSION}?
  • without "bugfix/" or "feature/" feels wrong.
  • All these unquoted variables hurt my eyes. Yeah, shell scripting is hard :/ Our preferred/standard way to write variables in such documents is ${VARIABLE:?} which is basically equivalent to set -u ; "$VARIABLE" i.e. an error will be raised if whoever pastes these command lines has not set $VARIABLE (mistakes do happen :)

Fixed.

#17 Updated by segfault 30 days ago

  • Status changed from Confirmed to In Progress

#18 Updated by intrigeri 29 days ago

  • Assignee changed from intrigeri to segfault
  • % Done changed from 0 to 80

Great! I've fixed with 8e03989e38f4b194290152a15f28df7815ecb91e a regression that had been introduced by the commit about quoting and merged into master. Please review my fix.

#19 Updated by segfault 28 days ago

  • Assignee changed from segfault to intrigeri
  • QA Check changed from Ready for QA to Pass

intrigeri wrote:

Great! I've fixed with 8e03989e38f4b194290152a15f28df7815ecb91e a regression that had been introduced by the commit about quoting and merged into master. Please review my fix.

Done, thanks for fixing this

#20 Updated by intrigeri 28 days ago

  • Status changed from In Progress to Fix committed
  • Assignee deleted (intrigeri)
  • % Done changed from 80 to 100

We're done here => doing the paperwork.

#21 Updated by intrigeri 28 days ago

  • Status changed from Fix committed to In Progress

#22 Updated by intrigeri 28 days ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF