Project

General

Profile

Feature #6397

Support booting from USB devices exposed as non-removable

Added by intrigeri over 5 years ago. Updated almost 2 years ago.

Status:
In Progress
Priority:
Normal
Assignee:
-
Category:
Hardware support
Target version:
-
Start date:
06/30/2014
Due date:
% Done:

10%

QA Check:
Feature Branch:
feature/6397-stop-relying-on-the-removable-bit
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:

Description

Tails refuses to boot from devices that expose themselves as non-removable. Historically, all USB sticks said they were removable, even if this is not correct according to the specification (removable is rather for devices that can be fed with removable medium, such as a floppy drive). Recently, quite a lot of new USB sticks (especially Sandisk) expose themselves as non-removable.

A simple plan could be to:

  1. pass the syslinux SYSAPPEND variable a bitmask that enables FSUUID= functionality (bit 0x40000): this feature appends, to the kernel command-line, the UUID of the partition the kernel is booted from (added in 6.03-pre9, see commit 386b59e1 in syslinux Git)
  2. add support to live-boot to search the SquashFS only on the filesystem specified by the FSUUID kernel command-line parameter, when present; when present, FSUUID should override any live-media= found on the command-line, so that we would still pass live-media=removable, that would only be taken into account when booting from DVD.

If this doesn't work nicely, the backup plan could be:

  1. Have Tails Installer write down the UUID in boot loader config on initial install
  2. Have Tails Installer preserve the UUID in boot loader config on upgrade
  3. Have Tails Updater (IUK) preserve the UUID in boot loader config on upgrade
  4. Require booting from the UUID that is in boot loader config, that is add such an option to live-boot
  5. Stop requiring the removable flag at boot time
  6. Only allow using persistent volume fr om the boot medium, that is add such an option to live-persist and/or live-boot

Note that live-boot already knows how to check an UUID that live-build embeds in the initramfs, so this should not be too hard.

Once done, the "known issues" page should be updated to remove all the Sandisk sticks, and possibly others.


Subtasks

Feature #7475: Have live-boot honor FSUUID=Confirmed


Related issues

Related to Tails - Feature #6641: Allow booting from USB and SDIO, regardless of the removable flag Rejected 02/08/2014
Related to Tails - Bug #6976: Document that Sandisk Cruzer Orbit 32gb stick hangs at installation time Resolved 03/25/2014
Related to Tails - Feature #8422: Support running Tails from internal hard drives Confirmed 12/12/2014
Related to Tails - Bug #15989: Update our plans to remove removable flag requirement Confirmed 09/28/2018
Blocked by Tails - Feature #7173: Upgrade to syslinux 6.03-pre18 or later Resolved 05/08/2014

History

#1 Updated by sajolida over 5 years ago

If I understand correctly, this plan should also correct some weird behavior that are sometimes experienced while starting Tails with several Tails devices plugged in (DVD + USB, or 2 USB). If not, then this might be worth being taken into consideration.

#2 Updated by intrigeri over 5 years ago

If I understand correctly, this plan should also correct some weird behavior that are
sometimes experienced while starting Tails with several Tails
devices plugged in

This is correct.

#3 Updated by intrigeri over 5 years ago

  • Target version set to Sustainability_M1

#4 Updated by BitingBird about 5 years ago

  • Related to Bug #6976: Document that Sandisk Cruzer Orbit 32gb stick hangs at installation time added

#5 Updated by intrigeri almost 5 years ago

  • Description updated (diff)

#6 Updated by intrigeri almost 5 years ago

Is it possible for something running in the initramfs to know for sure what filesystem the initramfs was loaded from (by the bootloader)? If it is, then we don't need the bootloader to provide us this information in any way.

#7 Updated by intrigeri almost 5 years ago

  • Description updated (diff)

#8 Updated by intrigeri almost 5 years ago

  • Feature Branch set to feature/6397-stop-relying-on-the-removable-bit

#9 Updated by intrigeri almost 5 years ago

  • Related to Feature #7173: Upgrade to syslinux 6.03-pre18 or later added

#10 Updated by intrigeri almost 5 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

Tested the branch, works fine (I can see FSUUID=1D15-3984 at the end of the kernel command-line). Note that no FSUUID is appended when booting from DVD.

#11 Updated by intrigeri almost 5 years ago

  • Description updated (diff)

#12 Updated by intrigeri almost 5 years ago

  • Assignee set to intrigeri

#13 Updated by BitingBird almost 5 years ago

When done, the "known issues" page should be updated to remove all the sandisk sticks, and possibly others :)

#14 Updated by intrigeri almost 5 years ago

  • Description updated (diff)

#15 Updated by intrigeri over 4 years ago

  • Related to deleted (Feature #7173: Upgrade to syslinux 6.03-pre18 or later)

#16 Updated by intrigeri over 4 years ago

  • Blocked by Feature #7173: Upgrade to syslinux 6.03-pre18 or later added

#17 Updated by sajolida over 4 years ago

Already two people reported that newer SanDisk might have a different firmware that fixes this issue. But that still need confirmation.

After this, and as SanDisk was the main (if not only) vendor with that behavior, maybe this ticket can be set as low priority or closed...

#18 Updated by intrigeri over 4 years ago

After this, and as SanDisk was the main (if not only) vendor with that behavior,
maybe this ticket can be set as low priority or closed...

Agreed. Let's see what happens :)

#19 Updated by sajolida over 4 years ago

Confirmed, some new SanDisk USB sticks are advertised again as removable:

/sys/devices/pci0000:00/0000:00:12.2/usb1/1-1/1-1:1.0/host2/target2:0:0/2:0:0:0/block/sda/removable
1

#20 Updated by emmapeel over 4 years ago

New user report indicates that SanDisk Extreme 3.0 without the Windows 8 logo, just bought, has removable flag off...

#21 Updated by sajolida over 4 years ago

Actually, this report says that the removable flag is set to "0"
(unset). We want the devices to be announced as removable.

#22 Updated by intrigeri over 4 years ago

  • Assignee changed from intrigeri to sajolida
  • QA Check set to Info Needed

Dear frontdesk, how is it going on your side? Did you notice a substantial change in the rate of bug reports that can be workaround'd by removing live-media=removable, since Sandisk changed their practices?

#23 Updated by sajolida over 4 years ago

Yes, definitely.

Still we got one report from someone who bought a new SanDisk without
the Windows 8 certificate and that wasn't flagged as removable. We check
this in /sys/devices and all.

Maybe this stick was a mislabeled from factory...

#24 Updated by sajolida over 4 years ago

  • Assignee changed from sajolida to intrigeri
  • QA Check deleted (Info Needed)

#25 Updated by sajolida over 4 years ago

I bought a SanDisk the other day. I was marked as "Software Compatibility: Windows 7, Windows 8" but it didn't have a Windows 8 logo or certification badge on it. Still it was correctly flagged as removable and boots fine.

#26 Updated by emmapeel over 4 years ago

Today a user reported not being able to boot unless removing the line "live-media=removable" at boot, though maybe External Hard Drives are unsupported. It happenned with a Western Digital Elements USB 3.0 External HDD.

#27 Updated by sajolida over 4 years ago

Emma: I think that this is expected for external hard disk.

#28 Updated by goupille over 4 years ago

maybe that's a bad news, but we have a report that this issue is also affecting SD-Cards, at least the Sony SDHC 8 GB (Type: SF-8UY/T1)

#29 Updated by BitingBird over 4 years ago

goupille: this should be documented in Known issues, do you care to open a ticket ?

#30 Updated by intrigeri over 4 years ago

goupille: this should be documented in Known issues, do you care to open a ticket ?

Note that this might be caused to the SD Card reader, rather than to the SD Card itself.

#31 Updated by WhiteWind over 4 years ago

Hello. Just wanted to say, I got a bought-new Sandisk Extreme 64GB USB 3.0 thumb drive (SDCZ80-064G-G46), there is no Windows 8 logo on the box, and I can't boot from it. BUT that is currently only my first try with something different than Windows, so that could be something else I've not done/ I did wrong, on my end. I have followed the instructions given on Tails mainsite, SecureBoot is disabled and the thumb drive is set as the boot drive in the UEFI. When I boot on it, I have a black screen with "No configuration file found. Boot: ". I would try the live-media= thing, but I'm not too confident, and well I don't really understand how it is done.

#32 Updated by intrigeri over 4 years ago

wrote (13 Jan 2015 23:42:52 GMT) :

Hello. Just wanted to say, I got a bought-new Sandisk Extreme 64GB USB 3.0 thumb drive (SDCZ80-064G-G46), there is no Windows 8 logo on the box, and I can't boot from it.

Please report a bug, including the answers to all questions on https://tails.boum.org/doc/first_steps/bug_reporting/tails_does_not_start/. Thanks!

#33 Updated by sajolida over 4 years ago

The live-media= parameter has to be entered on the boot screen, see:

https://tails.boum.org/doc/first_steps/startup_options#index1h1

So if you don't get to the boot screen, then I think that you are facing
a different issue. Feel free to contact our support team for more help.

#34 Updated by emmapeel about 4 years ago

New user reports problems with a brand new SanDisk Extreme USB3.0 stick with no Win8 logo.

User needed to remove the boot option to be able to boot.

#35 Updated by BitingBird about 4 years ago

I sent a patch to add SanDisk Extreme and Sony SD Card to the known issues. Further unsupported devices should get a dedicated doc ticket, instead of cluttering this one, which is about the code to solve the issue :)

#36 Updated by BitingBird about 4 years ago

goupille: regarding the SD Card, could you ask this person to run the following command:

sudo find /sys/devices -path "*sd*/removable" -print -exec cat '{}' \;

This returns the status of the removable flag on the device.

#37 Updated by sajolida about 4 years ago

Frontdesk people, please make sure to submit patches to add the relevant hardware on the known issues page. This is definitely on your plate as stated in https://tails.boum.org/contribute/working_together/roles/front_desk.

Also, please make sure to have people run that find command before adding hardware. It's in the frontdesk repo. As you surely know, people do all kind of crazy stuff to get Tails to run and report them in very funny ways sometimes...

#38 Updated by intrigeri about 4 years ago

It's getting hard to find the relevant information in here, so let's make things clear.

Frontdesk people: it's useful to report general tendencies (or even stats), as I asked on #6397#note-22: this will help us decide how urgent this problem is. (Looking at the history of the known issues section about it may actually be enough to assert its impact, as long as you keep it updated.)

However, this ticket is not about compiling a list of individual devices that can't boot with live-media=removable, nor about updating the known issues page (there are better places for that anyway). It's about fixing this problem for good.

Please, let's stop adding to the (already big) pile of comments anything that doesn't help fixing the underlying problem. Thanks!

#39 Updated by sajolida over 3 years ago

  • Target version deleted (Sustainability_M1)

#40 Updated by intrigeri over 3 years ago

  • Assignee deleted (intrigeri)

#41 Updated by intrigeri about 2 years ago

  • Related to Feature #8422: Support running Tails from internal hard drives added

#42 Updated by intrigeri about 2 years ago

https://sources.open-infrastructure.net/software/system-boot/commit/?id=c6de5da6c4b3e6744cc05601930508472de54549 makes system-boot (a live-boot fork) ignore the "removable" bit, and instead use heuristics based on the bus that's used (i.e. something relatively similar to what we do in Tails Installer and Tails Persistence Setup, where we treat all deviced plugged via USB or SDIO as OK). Might be good enough for us. I had rejected this idea on #6641 but I think my reasons were too simplistic: as the aforementioned commit shows, there's a sane way to do it. Now, this addresses the initial goal of this ticket, but it doesn't address the UX issue one has when starting a computer with 2 Tails devices plugged in — however, #7475#note-7 does, and can nicely combined with the solution described here.

#43 Updated by u almost 2 years ago

  • Subject changed from Support USB devices exposed as non-removable to Support booting from USB devices exposed as non-removable

#44 Updated by u 8 months ago

  • Related to deleted (Feature #6766: Display an error message when boot fails)

#45 Updated by u 8 months ago

  • Related to Feature #6766: Display an error message when boot fails added

#46 Updated by u 8 months ago

  • Related to deleted (Feature #6766: Display an error message when boot fails)

#47 Updated by u 8 months ago

  • Blocks Feature #6766: Display an error message when boot fails added

#48 Updated by u 8 months ago

  • Blocks deleted (Feature #6766: Display an error message when boot fails)

#49 Updated by intrigeri 7 months ago

#50 Updated by intrigeri 7 months ago

#51 Updated by intrigeri 7 months ago

  • Related to Bug #15989: Update our plans to remove removable flag requirement added

Also available in: Atom PDF