Project

General

Profile

Bug #11387

Tails upgrader left garbage in Tails partition

Added by mawo over 3 years ago. Updated over 3 years ago.

Status:
Confirmed
Priority:
Low
Assignee:
-
Category:
-
Target version:
-
Start date:
04/29/2016
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Upgrader

Description

When Tails Upgrader fails or is interrupted while doing an upgrade, the USB stick might end up with:

  • Some garbage in it, like /lib/live/mount/medium/tmp/tmp.SEbvQlhqGJ/tar. For example, if the upgrade failed during the download. It would be nice to remove this garbage as it otherwise forces the user to perform a full upgrade sooner than expected.
  • A USB stick in a weird state between the old and the new version if the upgrade failed while applying the upgrade, with possible security implications. It would be nice to prevent such a USB stick to even start. For example, by failing safe in Tails Greeter and display an error message.

In both cases ("failed while downloading" and "failed while applying upgrade") an error message is displayed to the user, but this doesn't take into account that the computer might be turned off by mistake before the error message.

Both improvements could be implemented using flags, added and removed in the system partition when doing the upgrade.


Related issues

Related to Tails - Bug #7073: Cloning Tails from installer copies all files from the live partition Confirmed 04/12/2014

History

#1 Updated by mawo over 3 years ago

meanwhile expired paste.debian.net content:

amnesia@amnesia:~$ du -h /lib/live/mount/medium
12K /lib/live/mount/medium/.disk
2.1M /lib/live/mount/medium/EFI/BOOT/grub/i386-efi
2.1M /lib/live/mount/medium/EFI/BOOT/grub
4.0M /lib/live/mount/medium/EFI/BOOT
4.0M /lib/live/mount/medium/EFI
1.6G /lib/live/mount/medium/live
316M /lib/live/mount/medium/tmp/tmp.SEbvQlhqGJ/tar
316M /lib/live/mount/medium/tmp/tmp.SEbvQlhqGJ
316M /lib/live/mount/medium/tmp
196K /lib/live/mount/medium/utils/linux
8.0K /lib/live/mount/medium/utils/mbr
244K /lib/live/mount/medium/utils/win32
452K /lib/live/mount/medium/utils
536K /lib/live/mount/medium/syslinux
8.0K /lib/live/mount/medium/System Volume Information
1.9G /lib/live/mount/medium/

amnesia@amnesia:~$ du -h /lib/live/mount/medium/live/*
114M /lib/live/mount/medium/live/2.0.1.squashfs
73M /lib/live/mount/medium/live/2.2.1.squashfs
277M /lib/live/mount/medium/live/2.2.squashfs
52K /lib/live/mount/medium/live/filesystem.packages
1.1G /lib/live/mount/medium/live/filesystem.squashfs
16M /lib/live/mount/medium/live/initrd2.img
16M /lib/live/mount/medium/live/initrd.img
4.0K /lib/live/mount/medium/live/Tails.module
2.7M /lib/live/mount/medium/live/vmlinuz
3.0M /lib/live/mount/medium/live/vmlinuz2

#2 Updated by sajolida over 3 years ago

  • Assignee set to intrigeri
  • QA Check set to Info Needed

The .squashfs files are definitely not garbage but the upgrade themselves :)

The tmp. look more like garbage.

Assigning this to intrigeri who's the maintainer of Tails Upgrader to see if we should include some more garbage collection code in it.

#3 Updated by intrigeri over 3 years ago

  • Status changed from New to Rejected

316M /lib/live/mount/medium/tmp/tmp.SEbvQlhqGJ/tar

Thanks!

When an upgrade succeeds, Tails Upgrade deletes those files. So, one of the previous upgrades (plausibly the one to 2.2, but only inspection of the mtime of these temporary files left behind would tell us for sure) failed. When this happened, Tails Upgrader should have displayed some error message, such as "An error occured while installing the upgrade. Your Tails device needs to be repaired and might be unable to restart. Please follow the instructions at file:///usr/share/doc/tails/website/doc/upgrade/error/install.en.html".

So we have several potential problems:

  • Why did that upgrade fail? Presumably, it's now way too late to investigate, so we can only give up for now, until we get a bug report with debugging info about that problem, and then it'll be for another ticket.
  • Did the user miss the error message? Or is there a bug in Tails Upgrader that made the error message not appear? Given this happened, to the latest, when upgrading to 2.2, presumably it's too late to be 100% certain of what happened back then. So again, we'll need a bug report with debugging info, prepared during the session in which such a failed upgrade happened, before we can act on it.

To conclude, on this ticket the best I could do if I left it open would be to retitle it "Tails Upgrader might sometimes not warn the user when an upgrade fails", leaving it in New (unconfirmed) state. IMO such a ticket would be mostly useless, so I am closing this ticket as non-actionnable. Please reopen once the situation with leftovers on the system partition happens again, and you can provide more information, so that we can know whether the upgrade failed (and why!), whether the expected error message is displayed, and whether the Upgrader actually leaves garbage behind in some specific situation.

#4 Updated by intrigeri over 3 years ago

  • Assignee deleted (intrigeri)

#5 Updated by sajolida over 3 years ago

  • Related to Bug #7073: Cloning Tails from installer copies all files from the live partition added

#6 Updated by sajolida over 3 years ago

By "some more garbage collection" I meant that even if an upgrade fails (think about a computer being shut down by mistake while doing an upgrade), this garbage should be removed after the fact. Maybe by Tails Upgrader when doing the next upgrade or at least by Tails Installer not to propage this garbage to other USB sticks when closing. This would be #7073, so I'll comment there as well as a legit scenario.

#7 Updated by intrigeri over 3 years ago

By "some more garbage collection" I meant that even if an upgrade fails (think about a computer being shut down by mistake while doing an upgrade), this garbage should be removed after the fact. Maybe by Tails Upgrader when doing the next upgrade or at least by Tails Installer not to propage this garbage to other USB sticks when closing.

When an upgrade fails, we tell the user that the Tails device must be reinstalled, and we have good reasons to do that: it may be running some half-working, and possibly dangerous, combination of two versions of Tails, and it should not be used anymore as-is. So, going through "the next upgrade" or cloning that device to another stick still look like situations we explicitly don't support, for good reasons.

If someone really wants to open that can of worms and try to support these situations better, fine by me, and I (seriously) wish you good luck. Sadly, at first glance, an easy solution like the ones proposed here will fix some of the incidental problems, but not the most important one :/

#8 Updated by intrigeri over 3 years ago

I think I must be looking at this from the wrong side, which might explain why I react so strongly and can't understand your point. Sorry about that. I'd love to understand better.

#9 Updated by sajolida over 3 years ago

Thanks for the additional explanation. This is something I was not aware of.

When an upgrade fails

You mean "after an upgrade failed"? or is this explained before starting
the upgrade? I'm not doing IUKs often so I don't know. Because again,
what happens if my battery dies out or if I experience a power cut while
doing the upgrade?

If our users can be at risk when using an USB stick that experienced a
failed or interrupted upgrade, then I don't think that an error message,
even if displayed before starting the upgrade, is enough.

We should instead make the resulting USB stick not boot at all. Couldn't
we add a flag somewhere when starting the upgrade and remove it only
once the upgrade is successful? And if this flag is present when booting
the USB stick, then we display an error message even before Tails Greeter?

#10 Updated by intrigeri over 3 years ago

  • Status changed from Rejected to Confirmed

When an upgrade fails

You mean "after an upgrade failed"?

Yes.

Because again, what happens if my battery dies out or if I experience a power cut while doing the upgrade?

OK, got it now, thanks! Indeed, in this case we'll have garbage on the system partition, and then there are two options:

  • the Upgrader was merely downloading stuff, and then the stick is still running the old version, life goes on and it's harmless to delete the temporary files;
  • the Upgrader had already partially applied the upgrade, and then the stick is running the (potentially dangerous) half-old/half-new Tails combination I was previously arguing about.

If our users can be at risk when using an USB stick that experienced a failed or interrupted upgrade, then I don't think that an error message, even if displayed before starting the upgrade, is enough.

We should instead make the resulting USB stick not boot at all.

I agree with this stance, at least in theory => reopening.

To sum up, we have two problems:

  • if an upgrade fails while downloading, then some space will be uselessly occupied on the stick (and all its future clones), so the user will have to manually upgrade this stick (and its clones) earlier than what would be ideal;
  • if an upgrade fails while it's being applied, the stick is in a potentially dangerous state, and we should communicate this better to the user, than what we currently do (i.e. telling them to re-install at the end of the failed upgrade, which only works if the system is still running, and not when power is shut down during the upgrade process).

Then we can discuss whether (one, or two of) the two problems we have are important enough to warrant prioritizing development efforts towards addressing it. I'm not sure personally.

The 1st one feels like a nice UX optimization, but given the low impact of the failure mode (no security problem, no data loss), I would treat it as "patches welcome".

The 2nd one feels like more important, as the user may use a Tails system that's in a weird (and potentially dangerous) shape, without knowing about it.

Thankfully, a solution to the 2nd problem also gives us relatively easy means to solve the 1st one :)

Couldn't we add a flag somewhere when starting the upgrade and remove it only once the upgrade is successful? And if this flag is present when booting the USB stick, then we display an error message even before Tails Greeter?

Excellent idea. We could add a flag when we start downloading the upgrade, and replace it with another flag when we start applying the upgrade. And then:

  • next time the Upgrader is used to upgrade, if there's the "downloading" flag, it can silently delete temporary files left from previous failed upgrade attempt; the Installer should also skip these files when cloning the currently running system, because it might happen before next upgrade; and no, we can't delete these files at any other time than when upgrading, since otherwise the system partition is mounted read-only, and we can't safely remount it read-write;
  • on boot, as you say we can detect early if the "applying upgrade" flag is present and display an error message (I would do it in the Greeter itself, instead of displaying its usual GUI).

Note that some of the design ideas we had on https://tails.boum.org/blueprint/Endless_upgrades/ (at least the Chrome OS one) are resistant to failed upgrades, and even support the case when a successfully upgraded system fails to boot. But let's not overengineer this now nor block on the Next Big Thing™ that might happen in 6 months... or in 4 years: we're too "good" at that, and here there's a relatively straightforward solution to the problem at hand (I'm not saying it's a cheap one, but it's a simple matter of programming in the Greeter and Upgrader, no additional complex design work is required).

#11 Updated by sajolida over 3 years ago

  • QA Check deleted (Info Needed)

Woh! I like this, thanks! I initially thought of having only "one flag to catch them all" but your solution is even nicer in terms of UX by preventing unnecessary panic modes on "failed while downloading".

I also agree that "failed while downloading" is clearly less important; and luckily maybe more frequent than "failed while applying upgrade" if we consider that downloading takes more time than applying (I have no data to support this claim).

I also like the idea of moving the warning to the Greeter which gives us some more room for having a nicer message, a "Power off" button, etc.

Then, regarding how much priority we should give to this issue, I'm not sure either. Supporting better "failed while downloading" clearly sounds low prio. Failing safe on "failed while applying upgrade" sounds more important as it can have security implications but it's also quite a corner case and almost everything else we already have on our busy roadmap have security implications one way or another :)

But since this has to do with Tails Upgrader I think you should have the final say on how important this sounds to you, maybe in terms of quantity of work, skills required to find someone else than you to work on this, etc. But I'm glad that we came up with a rational plan.

In terms of Redmine maybe, we should create a separate ticket for both cases ("failed while downloading" and "failed while applying upgrade"), maybe as children of this ticket. But you're my Redmine semantics guru so you probably have a better solution already :)

#12 Updated by intrigeri over 3 years ago

  • Assignee set to sajolida

Then, regarding how much priority we should give to this issue, I'm not sure either. Supporting better "failed while downloading" clearly sounds low prio. Failing safe on "failed while applying upgrade" sounds more important as it can have security implications but it's also quite a corner case and almost everything else we already have on our busy roadmap have security implications one way or another :)

I'm glad we're on the same page :)

But since this has to do with Tails Upgrader I think you should have the final say on how important this sounds to you, maybe in terms of quantity of work, skills required to find someone else than you to work on this, etc.

Regarding "failed while applying upgrade":

  • Importance: as said above, that's quite a corner case, and there are plenty of ways to make a much bigger difference, affecting all or most Tails users, in other areas.
  • Skills and amount of work: it's probably a few days of work for someone who's at ease with the technologies used in the Upgrader (Modern Perl with lots of Moose high-level constructs) and in the Greeter (Python + GTK), but even if this task feels easy to me, someone who just discovers these two codebases will probably need a few extra days to find their way in them, learn how to test their changes, and how to deliver their work. So they'd better be excited at this idea, and be willing to further contribute to these programs after this task is done. And of course, the Greeter and the Upgrader bits can be split between 2 people, with different sets of skills.

In terms of Redmine maybe, we should create a separate ticket for both cases ("failed while downloading" and "failed while applying upgrade"), maybe as children of this ticket. But you're my Redmine semantics guru so you probably have a better solution already :)

I would not bother before we have people willing to do the work. But updating this ticket's title and description seems important, as currently it doesn't reflect our best plan. Can you please do it?

#13 Updated by sajolida over 3 years ago

  • Description updated (diff)
  • Category deleted (Installation)
  • Assignee deleted (sajolida)
  • Priority changed from Normal to Low
  • Affected tool set to Upgrader

Also available in: Atom PDF