Project

General

Profile

Bug #16748

create-usb-image-from-iso uses the wrong version of syslinux

Added by anonym 5 months ago. Updated 4 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Installation
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
bugfix/16748-use-syslinux-from-chroot
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

(This is only about my local runs since the results on Jenkins are broken because earlier steps fail, and this feature has inter-scenario dependencies.)

The step "Booting Tails from a USB drive upgraded from USB with persistence enabled" fails with booting looking like this:

I've causally looked at the volume in a VM, and it has the appropriate partitions, so only the boot bits seem to have been corrupted (not investigated that at all yet).

I cannot reproduce this manually: I installed to USB, cloned the volume, then booted one of them and upgraded the other, and that worked fine, which is essentially what the test suite does (it doesn't copy, but redoes the same exact installation (we don't supply an --old-iso so the same ISO will be used) so it shouldn't matter).

booterror.png View (6.5 KB) anonym, 05/23/2019 12:55 PM


Related issues

Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision 21182abe (diff)
Added by segfault 5 months ago

create-usb-image-from-iso: Use syslinux from chroot (refs: #16748)

We used the syslinux from the vagrant box before, which caused issues
with when building Tails/Buster with a Stretch vagrant box and then
cloning the image via Tails Installer with syslinux from Buster.

Revision 6e097195 (diff)
Added by segfault 5 months ago

create-usb-image-from-iso: Use syslinux from chroot (refs: #16748)

We used the syslinux from the vagrant box before, which caused issues
with when building Tails/Buster with a Stretch vagrant box and then
cloning the image via Tails Installer with syslinux from Buster.

Revision 91ac010f
Added by anonym 5 months ago

Merge remote-tracking branch 'origin/bugfix/16748-use-syslinux-from-chroot' into feature/buster (refs: #16748)

Revision c9bd2f29 (diff)
Added by segfault 5 months ago

create-usb-image-from-iso: Add comment and variable prefix (refs: #16748)

Revision 1bffb90c
Added by anonym 5 months ago

Merge remote-tracking branch 'origin/bugfix/16748-use-syslinux-from-chroot' into stable

Fix-committed: #16748

History

#1 Updated by anonym 5 months ago

@segfault, @intrigeri,

Do you have any suggestions for how I can compare the "boot bits" of these volumes to try to narrow down what has happened?

Note that the Tails partitions are fine because the 'the running Tails is installed on USB drive "to_upgrade"' step succeeded, so the problem should be limited to the partition table/MBR/GPT header/similar. Oh, and to be clear, this is about legacy booting, not UEFI.

#2 Updated by anonym 5 months ago

anonym wrote:

Oh, and to be clear, this is about legacy booting, not UEFI.

So I tested UEFI boot, and that works. So it looks like the MBR (I guess?) is corrupted some how by the actions taken by the test suite.

#3 Updated by anonym 5 months ago

  • Description updated (diff)

#4 Updated by anonym 5 months ago

The problem is that the .img (used by the test suite to create the "old" install) is generated with the syslinux used by the builder VM (currently Stretch's) which might mismatch the version of syslinux we use in the installer (currently Buster's). So yes there is a mismatch. The fix is to make auto/scripts/create-usb-image-from-iso some how use the correct version of syslinux, e.g. the one used in live-build's chroot.

Is the chroot still available at the end of auto/build (where create-usb-image-from-iso is called)? If not, this might get awkward/difficult...

#5 Updated by anonym 5 months ago

  • Subject changed from Test suite cannot boot USB-to-USB upgraded Tails/Buster to create-usb-image-from-iso uses the wrong version of syslinux
  • Category changed from Test suite to Installation
  • Parent task deleted (#16281)

#6 Updated by segfault 5 months ago

  • Category changed from Installation to Test suite
  • Feature Branch set to bugfix/16748-use-syslinux-from-chroot

In a buildlog I see chroot/debootstrap still being accessed after live-build finished, so we might be good. I will just try using chroot/usr/bin/syslinux in create-usb-image-from-iso.

#7 Updated by segfault 5 months ago

  • Status changed from Confirmed to In Progress

#8 Updated by intrigeri 5 months ago

In a buildlog I see chroot/debootstrap still being accessed after live-build finished, so we might be good. I will just try using chroot/usr/bin/syslinux in create-usb-image-from-iso.

@segfault, note that the modules version has to match the syslinux binary version (there's no ABI compatibility, which is the reason for some of the contrived things we do in Tails Installer about this sort of things). So you probably also should copy the modules from the chroot.

#9 Updated by segfault 5 months ago

intrigeri wrote:

In a buildlog I see chroot/debootstrap still being accessed after live-build finished, so we might be good. I will just try using chroot/usr/bin/syslinux in create-usb-image-from-iso.

@segfault, note that the modules version has to match the syslinux binary version (there's no ABI compatibility, which is the reason for some of the contrived things we do in Tails Installer about this sort of things). So you probably also should copy the modules from the chroot.

ACK, will do that

#10 Updated by intrigeri 5 months ago

#11 Updated by anonym 5 months ago

  • Category changed from Test suite to Installation
  • Assignee set to anonym

To me the least error prone approach would be to the these things inside the chroot instead. I'll give it a quick try, and pass this ticket to segfault if I fail.

#12 Updated by anonym 5 months ago

  • Assignee changed from anonym to segfault

anonym wrote:

To me the least error prone approach would be to the these things inside the chroot instead. I'll give it a quick try, and pass this ticket to segfault if I fail.

I tested:

--- a/auto/build
+++ b/auto/build
@@ -181,4 +181,8 @@ echo "Generating build manifest..." 
 generate-build-manifest chroot/debootstrap "${BUILD_MANIFEST}" 

 echo "Creating USB image ${BUILD_USB_IMAGE_FILENAME}..." 
-create-usb-image-from-iso "${BUILD_ISO_FILENAME}" 
+cp auto/scripts/create-usb-image-from-iso chroot
+chmod a+x chroot/create-usb-image-from-iso
+mv "${BUILD_ISO_FILENAME}" chroot
+chroot chroot /create-usb-image-from-iso "${BUILD_ISO_FILENAME}" 
+mv chroot/"${BUILD_ISO_FILENAME}" .

but that failed with:

[...]
INFO:__main__:Setting up loop device
Traceback (most recent call last):
  File "/create-usb-image-from-iso", line 340, in <module>
    main()
  File "/create-usb-image-from-iso", line 336, in main
    image_creator.create_image()
  File "/create-usb-image-from-iso", line 100, in create_image
    with self.setup_loop_device():
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/create-usb-image-from-iso", line 149, in setup_loop_device
    with self.get_udisks_client() as udisks_client:
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/create-usb-image-from-iso", line 93, in get_udisks_client
    client = UDisks.Client().new_sync()
gi.repository.GLib.Error: g-io-error-quark: Could not connect: No such file or directory (1)

so I don't think my approach is "less error-prone" any more. :) Nevermind!

#13 Updated by anonym 5 months ago

I merged the current state of bugfix/16748-use-syslinux-from-chroot into feature/buster because it indeed fixes the original problem so now we can test beyond those failures.

#14 Updated by segfault 5 months ago

anonym wrote:

I merged the current state of bugfix/16748-use-syslinux-from-chroot into feature/buster because it indeed fixes the original problem so now we can test beyond those failures.

Cool! So, what's the status here? fix committed? And what about stable? This didn't hurt us yet because the vagrant box has the same syslinux version as Tails 3.x, and this will probably stay that way, but we could still merge this into stable.

#15 Updated by anonym 5 months ago

  • % Done changed from 0 to 40
  • QA Check set to Dev Needed

segfault wrote:

anonym wrote:

I merged the current state of bugfix/16748-use-syslinux-from-chroot into feature/buster because it indeed fixes the original problem so now we can test beyond those failures.

Cool! So, what's the status here? fix committed?

Let's finish up with adding a CHROOT_ prefix to SYSLINUX_COM32MODULES_DIR for consistency, and also add a comment above those two vars about why we use the chroot.

And what about stable? This didn't hurt us yet because the vagrant box has the same syslinux version as Tails 3.x, and this will probably stay that way, but we could still merge this into stable.

Definitely!

#16 Updated by segfault 5 months ago

  • Assignee changed from segfault to anonym
  • QA Check changed from Dev Needed to Ready for QA

anonym wrote:

Let's finish up with adding a CHROOT_ prefix to SYSLINUX_COM32MODULES_DIR for consistency, and also add a comment above those two vars about why we use the chroot.

Done

#17 Updated by anonym 5 months ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 40 to 100

#18 Updated by anonym 5 months ago

  • Assignee deleted (anonym)
  • QA Check changed from Ready for QA to Pass

LGTM!

#19 Updated by intrigeri 5 months ago

  • Target version changed from Tails_4.0 to Tails_3.15

(This was merged into stable.)

#20 Updated by intrigeri 4 months ago

  • Status changed from Fix committed to Resolved
  • Target version changed from Tails_3.15 to Tails_3.14.1

Also available in: Atom PDF