Project

General

Profile

Bug #15477

Consider upgrading to live-boot 1:20180328+

Added by intrigeri over 1 year ago. Updated 4 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
03/29/2018
Due date:
% Done:

10%

Estimated time:
4.00 h
Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

In particular, https://bugs.debian.org/886328 ("use /run/live instead of /lib/live/mount") might affect all kinds of stuff such as memory erasure, live-persist, and more. OTOH it might allow us to drop some of our delta e.g. config/chroot_local-patches/live-boot:_dont_mount_live_overlay_twice.patch.


Related issues

Blocks Tails - Bug #15146: Make memory erasure feature compatible with overlayfs Confirmed 01/03/2018
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed 03/22/2019
Blocked by Tails - Feature #15944: Port Tails to Buster In Progress 09/12/2018

Associated revisions

Revision 8b452697 (diff)
Added by intrigeri about 1 year ago

Drop obsolete patch (refs: #15477)

live-boot's commit 0d878d3a679820d570a20c0fa1c1a2c5b92ad037, that's included in
live-boot 1:20180328, removes the call to mount our patch was removing.

Revision 5f906114 (diff)
Added by intrigeri about 1 year ago

Drop obsolete patch: the bug it workarounds only happens with CONFIG_AUFS_DEBUG enabled.

We disable CONFIG_AUFS_DEBUG in config/chroot_local-hooks/13-aufs
and the Debian package did it as well (Debian#886329).

Besides, that patch does not apply cleanly on current live-boot anymore
(refs: #15477).

Revision 928c868e (diff)
Added by intrigeri about 1 year ago

Adjust AppArmor aliases and tunables for mount points used by current live-boot (refs: #15477).

Revision df3c2dea (diff)
Added by intrigeri about 1 year ago

Update paths for mount points used by current live-boot (refs: #15477).

Revision 99a01f8a (diff)
Added by intrigeri about 1 year ago

Import tails-perl5lib feature/buster branch at commit 7a6700b as a patch (refs: #15477).

Revision 4377f177 (diff)
Added by intrigeri about 1 year ago

Remove AppArmor profile customization that's not needed with current live-boot (refs: #15477)

The /lib/** kind of rules don't conflict anymore with the ones generated
due to the AppArmor aliases we set up to support /run/live/*.

Revision 016e53e9 (diff)
Added by intrigeri about 1 year ago

Add a /run/live/persistence → /live/persistence symlink (refs: #15477)

The same rationale as 59573b6f7a91dc1f1f5dc9c123ba4f1e350388fb
applies.

Revision f365b9bf (diff)
Added by intrigeri 6 months ago

Revert incomplete and partly broken attempt to adjust to live-boot 1:20180328+ (refs: #15477)

I've uploaded Stretch's live-boot to our feature-buster APT overlay suite.
Let's install it and postpone finishing these adjustments. Accordingly, let's
revert our code to a state suitable for Stretch's live-boot.

This reverts commits 016e53e97bab7470010b05edbf00a3c2dbe539e7,
4377f177e080318db987b1832d93183bf915653d,
99a01f8a48c8f41982a502b0de0378405d796cde,
df3c2dea585f68d0b98a114fb663b32c08418fe0,
928c868eacbad146da4e9427135fff4957534d74
and 8b452697f8179fad87264579e7ebca7688070f38.

Revision 60cfde6b (diff)
Added by intrigeri 6 months ago

Refresh patch (refs: #15477)

History

#1 Updated by intrigeri over 1 year ago

Next steps:

  • check if this upgrade breaks anything in our automated test suite
  • git grep live/mount and adjust/investigate each occurence

#2 Updated by intrigeri over 1 year ago

  • Priority changed from Normal to High

This upgrade breaks the build of feature/buster because config/chroot_local-patches/live-boot:_dont_mount_live_overlay_twice.patch does not apply cleanly anymore.

#3 Updated by intrigeri about 1 year ago

  • Related to Bug #15146: Make memory erasure feature compatible with overlayfs added

#4 Updated by intrigeri about 1 year ago

  • Related to deleted (Bug #15146: Make memory erasure feature compatible with overlayfs)

#5 Updated by intrigeri about 1 year ago

  • Blocks Bug #15146: Make memory erasure feature compatible with overlayfs added

#6 Updated by intrigeri about 1 year ago

intrigeri wrote:

This upgrade breaks the build of feature/buster because config/chroot_local-patches/live-boot:_dont_mount_live_overlay_twice.patch does not apply cleanly anymore.

Fixed that part so now one can check what still works and what's broken.

#7 Updated by intrigeri about 1 year ago

  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri
  • % Done changed from 0 to 10

A nice side effect of the mountpoints not being under /lib anymore is that overlapping rules should be less of an issue with our AppArmor aliases so we should adjust our AppArmor profiles patches and update wiki/src/contribute/design/application_isolation.mdwn accordingly.

Next steps:

  • Address the same kind of issues that will now appear in different places e.g. /etc/apparmor.d/usr.sbin.cupsd: /{,var/}run/** rm, but that pattern is way less common that the one we were suffering from previously.
  • Check if the kludges done in apparmor-adjust-cupsd-profile.diff for #9963 are still needed.
  • apparmor-adjust-thunderbird-profile.diff: drop the kludges that were added due to conflicting rules.
  • update wiki/src/contribute/design/application_isolation.mdwn

Besides, persistence is broken because live-boot expects stuff to be mounted on /run/live/persistence while our live-persist script mounts it on /live/persistence. Previously it worked thanks to a symlink, see 59573b6f7a91dc1f1f5dc9c123ba4f1e350388fb for details. Given the amount of stuff we have that relies on the /live/persitence path, I think we should replace that old symlink (and the corresponding bits in auto/build) with another one created with new file in config/chroot_local-includes/usr/lib/tmpfiles.d/ and see if it's enough to fix things.

Also, config/chroot_local-patches/remount_persistence_filesystem_readonly_on_shutdown.patch applies just fine for that's merely because live-tools was not updated to the new paths yet, which can itself cause problems e.g. wrt. caching files before shutdown, ejecting the DVD, and the toram feature (that we don't support though).

#8 Updated by intrigeri about 1 year ago

intrigeri wrote:

Besides, persistence is broken because live-boot expects stuff to be mounted on /run/live/persistence while our live-persist script mounts it on /live/persistence. Previously it worked thanks to a symlink, see 59573b6f7a91dc1f1f5dc9c123ba4f1e350388fb for details. Given the amount of stuff we have that relies on the /live/persitence path, I think we should replace that old symlink (and the corresponding bits in auto/build) with another one created with new file in config/chroot_local-includes/usr/lib/tmpfiles.d/ and see if it's enough to fix things.

Adding such a symlink seems to break things even more: persistence cannot be unlocked anymore.

Also, automated tests for memory erasure on shutdown are broken: the message that tells memory wipe was completed is never displayed. I noticed these error messages (with plenty of SquashFS errors before and in between):

systemd-shutdown[1]: Failed to mount /sys to /run/initramfs/sys: No such file or directory
systemd-shutdown[1]: Failed to execute shutdown binary: Input/output error
systemd-shutdown[1]: Failed to finalize file systems, ignoring

Also, Evince manages to open /live/overlay/home/amnesia/.gnupg/default-testpage.pdf (wrong), but I see "EvinceUnableToOpen.png" after at most 10 seconds passes (wrong too) and then of course AppArmor has denied "/usr/bin/evince" from opening "/run/live/overlay/home/amnesia/.gnupg/default-testpage.pdf" fails (correct in this case, but not what it should be). Same problem for Watching a MP4 video stored on the non-persistent filesystem when opening stuff under /live/overlay/home/amnesia/.gnupg/. I think that's because we still ship a /live/overlay symlink and live-boot manages a /lib/live/mount bind-mount (mount --rbind /run/live ${rootmnt}/lib/live/mount) which provides alternative paths to files we want to deny access from. Even if we dropped our /live/overlay symlink, these files could still be accessed via /lib/live/mount/overlay/. So either we get rid of that bind-mount (and then we have to adapt live-tools) or we re-add the AppArmor aliases that point to the old location (cost: potentially slower boot due to AppArmor profile compilation).

#9 Updated by intrigeri 10 months ago

  • Assignee changed from intrigeri to CyrilBrulebois
  • Target version changed from Tails_4.0 to Tails_3.10.1
  • Estimated time set to 4.00 h

I'm sorry I started this work on feature/buster and left it in a quite broken state. Please spend some — but not too much — time trying to complete this and if it proves to be harder than that, revert to Stretch's live-boot and revert the commits I did for this ticket.

#10 Updated by intrigeri 10 months ago

#11 Updated by intrigeri 10 months ago

#12 Updated by intrigeri 9 months ago

  • Target version changed from Tails_3.10.1 to Tails_3.11

#13 Updated by CyrilBrulebois 8 months ago

  • Target version changed from Tails_3.11 to Tails_3.12

Punting to version 3.12, as 3.11 is close and other topics have higher priority (USB image etc.).

#14 Updated by CyrilBrulebois 7 months ago

#15 Updated by CyrilBrulebois 7 months ago

#16 Updated by CyrilBrulebois 7 months ago

  • Related to deleted (Feature #15507: Core work 2019Q1: Foundations Team)

#17 Updated by CyrilBrulebois 7 months ago

#18 Updated by CyrilBrulebois 7 months ago

Refreshing my memory by reading this again, I'm reminded that we have these tests that fail regularly on the devel branch already (seen in Jenkins):

Failing Scenarios:
cucumber features/erase_memory.feature:62 # Scenario: Erasure of the aufs read-write branch on shutdown
cucumber features/emergency_shutdown.feature:13 # Scenario: Tails erases memory on DVD boot medium removal: aufs read-write branch
cucumber features/emergency_shutdown.feature:22 # Scenario: Tails erases memory on DVD boot medium removal: vfat
cucumber features/emergency_shutdown.feature:33 # Scenario: Tails erases memory on DVD boot medium removal: LUKS-encrypted ext4
cucumber features/emergency_shutdown.feature:44 # Scenario: Tails erases memory and shuts down on USB boot medium removal: persistent data

Maybe we should investigate those at some point? From a quick search, some tests were mentioned in other bug reports already, like #13462.

#19 Updated by intrigeri 7 months ago

Refreshing my memory by reading this again, I'm reminded that we have these tests that fail regularly on the devel branch already (seen in Jenkins):

That's #16097, on my plate for 3.12, but I wanted to discuss it at the FT meeting today. Probably a blocker before we try to fix the other regression in these tests on feature/buster.

#20 Updated by intrigeri 7 months ago

  • Assignee changed from CyrilBrulebois to intrigeri
  • Target version changed from Tails_3.12 to Tails_4.0

#21 Updated by intrigeri 6 months ago

intrigeri wrote:

I'm sorry I started this work on feature/buster and left it in a quite broken state. Please spend some — but not too much — time trying to complete this and if it proves to be harder than that, revert to Stretch's live-boot and revert the commits I did for this ticket.

Four months later, since that's not fixed I'll do these reverts so this does not taint test suite results, which may hide other kinds of issues we're trying to identify.

#22 Updated by intrigeri 6 months ago

#23 Updated by intrigeri 6 months ago

  • Subject changed from Adjust for live-boot 1:20180328+ to Consider upgrading to live-boot 1:20180328+
  • Assignee deleted (intrigeri)
  • Priority changed from High to Low

Reverted to Stretch's live-boot for now, so let's make this ticket about possibly upgrading to Buster's live-boot again. Not a blocker for the 4.0 release in itself.

#24 Updated by intrigeri 5 months ago

#25 Updated by intrigeri 5 months ago

#26 Updated by intrigeri 4 months ago

#27 Updated by intrigeri 4 months ago

  • Priority changed from Low to Normal
  • Target version deleted (Tails_4.0)

Also available in: Atom PDF