Project

General

Profile

Bug #15987

Check the system partition on every boot and grow it if needed

Added by u 8 months ago. Updated 3 months ago.

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
09/28/2018
Due date:
% Done:

0%

Estimated time:
4.00 h
QA Check:
Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Related issues

Related to Tails - Feature #15319: Grow system partition during boot when started from USB Resolved 10/16/2018
Related to Tails - Feature #15293: Creating & preparing the disk image Resolved 02/17/2018
Related to Tails - Bug #16389: Some USB sticks become unbootable in legacy BIOS mode after first boot In Progress 01/25/2019

History

#1 Updated by u 8 months ago

  • Target version set to Tails_3.11

#2 Updated by u 8 months ago

  • Blocked by Bug #15991: Code review & rubber-duck for USB Image added

#3 Updated by intrigeri 8 months ago

  • Blocked by deleted (Bug #15991: Code review & rubber-duck for USB Image)

#4 Updated by intrigeri 8 months ago

AFAICT the current, WIP code only does this on first boot, so my understanding is that this is not a duplicate of #15319.

#5 Updated by segfault 7 months ago

What is the reasoning for doing this on every boot? Currently, the code checks if the partitioning script has been executed before (by comparing the GUID with the default value) and exits in that case.

#6 Updated by intrigeri 7 months ago

  • Assignee changed from segfault to intrigeri
  • QA Check set to Info Needed

#7 Updated by intrigeri 7 months ago

  • Assignee changed from intrigeri to segfault
  • QA Check deleted (Info Needed)

segfault wrote:

What is the reasoning for doing this on every boot?

Sadly, I don't remember and it does not seem to be explained anywhere. I've checked the Git history of fundraising.git but that line was in the initial version we've imported from the pad. So we're left with reverse engineering why we wrote that in the first place. I'm not sure whose job that is (probably not mine) but I fancied brainstorming it a bit and the only reasons I can easily think of are these ones:

  • The magic numbers (size) we're hardcoding in the partitioning script will probably change some day; that's simply how it is. Growing if needed on every boot gives us and the users who have no persistent volume an already existing migration path to the new settings.
  • Say I have installed Tails on a small USB stick and I'm not using persistence. After a while, I can't apply an automatic upgrade anymore because my system partition is too small, so I decide to migrate to a larger stick. For some reason, instead of installing a new Tails from scratch, I Google around and copy'n'paste a dd command line from Stack Overflow which creates a binary copy of my old Tails on the new stick. My new stick will presumably boot just fine, but it won't solve my problem at all, unless we do grow the system partition on every boot.

TBH none of those seem to be strong reasons to bother to me, which makes me wonder why we wrote this bullet point in the first place: at least 3-4 people have ACK'ed it and we must have had a good reason. Hopefully.

Note that this is not an explicit sponsor deliverable, so we're not tight by a contract and can use our best judgement without having to explain anyone why the plan changed. If we decide it's not worth the effort, fine, let's skip this. I'd like to make this decision with our team-mates though, they might remember or think about a good reason to do it :) Sadly, sajolida won't attend the next meeting so you'll need to get his input via different means. While doing so, please ensure he's aware of your time constraints.

#8 Updated by intrigeri 7 months ago

  • Related to Feature #15319: Grow system partition during boot when started from USB added

#9 Updated by segfault 7 months ago

  • Assignee changed from segfault to sajolida
  • QA Check set to Info Needed

sajolida, do you remember why this was added to the pad? The deadline for implementing this is Nov 27, and we will talk about it during our next team meeting (Wednesday Nov 7).

#10 Updated by intrigeri 7 months ago

sajolida, do you remember why this was added to the pad? The deadline for implementing this is Nov 27, and we will talk about it during our next team meeting (Wednesday Nov 7).

Other time constraints: my next round of code review is scheduled on Nov 12 and the hopefully final one is scheduled on Nov 22-23, so we need to make a decision sufficiently early so that segfault has time to implement whatever it is in time, ideally before the Nov 12 review.

#11 Updated by sajolida 7 months ago

  • Assignee changed from sajolida to segfault

I'm not sure either why we wrote this.

intrigeri's first point seems relevant: it sounds interesting to automatically grow the size of the system partition whenever we decide to use a bigger one. But I understand that this can't work for people with a persistence.

So maybe it depends on how much extra work it would be to also check for the size of the partition (in addition to checking the UUID as you do already) before deciding to enlarge it in the partition script.

From your #note-5 I understand that you are already checking the system partition on every boot and maybe that's what we meant here. The title of this ticket could then be: "Check the system partition on every boot and grow it if needed" which is vaguely similar and looks like what you're already doing.

#12 Updated by segfault 7 months ago

  • Target version changed from Tails_3.11 to Tails_3.12

#13 Updated by intrigeri 6 months ago

#14 Updated by u 5 months ago

  • Subject changed from Check & grow partition size on every boot to Check the system partition on every boot and grow it if needed

#15 Updated by u 5 months ago

intrigeri's first point seems relevant: it sounds interesting to automatically grow the size of the system partition whenever we decide to use a bigger one. But I understand that this can't work for people with a persistence.

I agree with both your reasoning.

But if this does not work for people using persistence, the cost-benefit ratio might be quite low. Sadly, we don't have statistics about using persistence.

So maybe it depends on how much extra work it would be to also check for the size of the partition (in addition to checking the UUID as you do already) before deciding to enlarge it in the partition script.

From your #note-5 I understand that you are already checking the system partition on every boot and maybe that's what we meant here. The title of this ticket could then be: "Check the system partition on every boot and grow it if needed" which is vaguely similar and looks like what you're already doing.

I've renamed the ticket.

@segfault: do you think this is something

- worth doing
- that you can do
- and within which time frame do you see yourself doing this?

As this is not a sponsor deliverable, we could re-schedule it.
And if we decide not to do that, we could reject this ticket.

What do you think?

#16 Updated by u 5 months ago

ping? your input is needed, see my last comment.

#17 Updated by u 5 months ago

ping?

#18 Updated by intrigeri 5 months ago

#19 Updated by segfault 5 months ago

Sorry for the late response when it was apparently wanted urgently - my understanding is that this is not a sponsor deliverable and not a blocker for the beta, so I didn't handle it has high priority.

I'm really not sure if the two use cases are worth the effort, so I would prefer to wait until late in the project and see if this still fits in the budget. On the other hand, a change like this could cause bugs which we might not want to introduce after the beta and extensive testing.

#20 Updated by u 5 months ago

  • QA Check deleted (Info Needed)

Agreed. Let's reevaluate this during a meeting.

#21 Updated by intrigeri 5 months ago

Agreed.

Same. I'll adjust the metadata accordingly.

#22 Updated by intrigeri 5 months ago

  • Target version deleted (Tails_3.12)

#23 Updated by u 4 months ago

  • Parent task deleted (#15293)

#24 Updated by u 4 months ago

  • Related to Feature #15293: Creating & preparing the disk image added

#25 Updated by adamantium 3 months ago

I see adjustment of partition sizes as a potentially problematic issue. If persistence exists, perhaps a nearly full volume, then an expansion of the system partition might be impossible or at least unwanted by the user. Automatic expansion just sounds like a bad idea in every case aside from a new USB stick install from a USB .img file using Disks.

Case in point: Had 3.11 with a persistent volume, tried to auto-upgrade to 3.12, but got a message that it failed and would need to be manually upgraded to repair, as the USB stick probably wouldn't boot and needed repair (it did require repair).

Relevant Backstory:Modified things awhile ago and did some manual backing up related to #10976 recovery before I completely trusted just reconfiguring persistence with the same password. I had about 12-13GB of persistent data. New policy with tails released Juneish 2018 changed the default system partition on a new usb stick to 8GB, but my stick was only 16GB, and an 8GB system partition prevented me from restoring a 12-13GB persistent volume. I scratched my head awhile, and ended up booting from a different Tails, and using GPartEd to resize the 8GB tails system partition on my stick of interest down to about 4GB. No problem. Problem was solved, I created a persistent volume on the remainder of the stick, and migrated my old persistence volume to it. The stick has worked great ever since.... until yesterday.

During an attempted 3.11 to 3.12 automatic upgrade, it failed as described 2 paragraphs ago. I booted from another (older) tails, and tried using tails installer and the 3.12 .iso to repair the Tails that broke. It didn't work, and acted like it would boot, but stalled on the bootloader countdown (UEFI grey background I believe). Never booted. After much research and head scratching, I eventually decided to install to a 3rd USB an intermediary Tails from the (now just released) 3.12.1 .img usb image using Disks. It completely wiped the usb, and thank you for clearly putting the warning that this would happen, as tails installer only wipes the tails system partition, not the persistence volume.

Finally I booted the intermediary 3rd USB stick (Tails 3.12.1), and using Tails installer, I cloned the 3.12.1 to the original stick that broke during the automatic upgrade. Success! Though it took me half a day to achieve it.

Though my efforts described in this message perhaps qualify me as a power user, I still am not a linux guru. My understanding is ambitious but partial. There seem to be some hiccups in the transition from the legacy .iso based upgrades to the .img USB based. I still don't completely see how this switch will eliminate the eventual periodic need for manual upgrades. I have a genuine curiousity for how all these things work together.

Additionally, THANK YOU tails developers for your work on making the most principled OS in existence IMHO. Tails makes more sense, and is more adaptive for most people (whether they are aware of it yetor not). Built on a solid ideological foundation, Tails remains both libre & gratis, and worthy of donations.

#26 Updated by u 3 months ago

  • Related to Bug #16389: Some USB sticks become unbootable in legacy BIOS mode after first boot added

Also available in: Atom PDF