Project

General

Profile

Bug #11873

"Upgrade from ISO" fails from 32-bit Tails with 64-bit ISO

Added by intrigeri about 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Installation
Target version:
Start date:
10/11/2016
Due date:
% Done:

100%

Feature Branch:
bug/11873+upgrade_32-bit_to_64-bit_ISO
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Installer

Related issues

Related to Tails - Feature #8183: Ship a 64-bit (x86_64) instead of 32-bit userspace Resolved 10/11/2016
Related to Tails - Feature #13513: Replace syslinux:i386 with syslinux:amd64 in the ISO9660 filesystem Resolved 07/25/2017
Blocked by Tails - Bug #11961: Drop obsolete multiarch handling in Stretch Resolved 11/18/2016

Associated revisions

Revision c2eb7bd6 (diff)
Added by u about 3 years ago

As the 32-bit executable will be executed by both 32 and 64 bit systems, we simply want to replace the 64 bit syslinux executable for now in order to allow upgrades to Tails Stretch (64-bits) from Tails Jessie (32-bits). In Tails 3.3 we can then revert this back and stop shipping a syslinux 32-bit. will-fix: #11873

Revision e478f4ee (diff)
Added by intrigeri about 3 years ago

Adjust comments to reflect what we're doing (refs: #11873).

Revision e373cb50 (diff)
Added by intrigeri about 3 years ago

Remove now unused variable (refs: #11873).

Revision a11b1223 (diff)
Added by intrigeri about 3 years ago

Add the architecture that's missing, not the one we already have (refs: #11873).

The previous code used to work thanks to another bug (#11961) that was
fixed since then.

Revision 21a1df43
Added by intrigeri about 3 years ago

Merge branch 'bug/11873+upgrade_32-bit_to_64-bit_ISO' into feature/stretch

refs: #11873

Revision febdcdc4 (diff)
Added by intrigeri almost 3 years ago

Install libc6-i386, needed for Tails Installer to run the syslinux:i386 from the ISO (refs: #11873).

History

#1 Updated by intrigeri about 3 years ago

  • Priority changed from Normal to Elevated
  • Parent task deleted (#8183)

#2 Updated by intrigeri about 3 years ago

  • Related to Feature #8183: Ship a 64-bit (x86_64) instead of 32-bit userspace added

#3 Updated by u about 3 years ago

  • Assignee changed from intrigeri to u

#4 Updated by u about 3 years ago

It would be nice, next time, to have a more detailed description of the bug report ;)

What I've found is that, when using "Upgrade from ISO" in Debian, there is no problem.
But in Tails, the initial upgrade works, but then syslinux is not found.


sync

sync

/tmp/tmpDWCAq2/syslinux  -d syslinux /dev/sdb1
/bin/sh: 1: /tmp/tmpDWCAq2/syslinux: not found

Es gab ein Problem beim Ausführen des folgenden Befehls »/tmp/tmpDWCAq2/syslinux  -d syslinux /dev/sdb1«. 
Eine detailreichere Fehlerbeschreibung ist hier zu finden: »/tmp/tails-installer-fyzX9f«
Tails installation failed!
Es gab ein Problem beim Ausführen des folgenden Befehls »/tmp/tmpDWCAq2/syslinux  -d syslinux /dev/sdb1«. 
Eine detailreichere Fehlerbeschreibung ist hier zu finden: »/tmp/tails-installer-fyzX9f«

This seems logical, because as far as i remember, correct me if I'm wrong, we don't include syslinux 64 bits in Tails i386.

Will investigate this further.

#5 Updated by u about 3 years ago

I tried this with a 2.7 ISO on a stick -> upgrading the 3.0 from the nightly build.

#6 Updated by intrigeri about 3 years ago

u wrote:

This seems logical, because as far as i remember, correct me if I'm wrong, we don't include syslinux 64 bits in Tails i386.

I think that's correct. Tails i386 would not be able to run a 64-bit syslinux anyway.

Maybe the problem is that we're not including a 32-bit syslinux in the 64-bit ISO (in Tails we're using syslinux found in the ISO, instead of the system one, no? Or is it the opposite? I don't remember, sorry!).

#7 Updated by u about 3 years ago

intrigeri wrote:

u wrote:

This seems logical, because as far as i remember, correct me if I'm wrong, we don't include syslinux 64 bits in Tails i386.

I think that's correct. Tails i386 would not be able to run a 64-bit syslinux anyway.

Maybe the problem is that we're not including a 32-bit syslinux in the 64-bit ISO (in Tails we're using syslinux found in the ISO, instead of the system one, no? Or is it the opposite? I don't remember, sorry!).

Yes you're correct, we use syslinux from the ISO. So we need to ship a 32-bit syslinux in the 64-bit ISO.
No modification needed in tails-installer probably.

#8 Updated by u about 3 years ago

  • % Done changed from 0 to 10

#9 Updated by u about 3 years ago

  • Feature Branch set to 451f:tails/bug/11873+upgrade_32-bit_to_64-bit_ISO

Tentative fix, totally untested.

#10 Updated by u about 3 years ago

  • Assignee changed from u to intrigeri
  • QA Check set to Ready for QA

I've tried to reapply ac29b779, inverting i386 and amd64. But I did not update the APT sources list part, as it looked to me like we already have activated i386 architecture, but I might be wrong.

Or do we need config/chroot_local-hooks/03-dpkg-architectures ?

#! /bin/sh
set -e
echo "Configuring dpkg architectures" 
dpkg --add-architecture i386

I let you check if this is sufficient to fix the issue.

#12 Updated by intrigeri about 3 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to u
  • QA Check deleted (Ready for QA)

#13 Updated by u about 3 years ago

Branch seems to build now, but I will have to test if syslinux-i386 is really included and if we can upgrade.

#14 Updated by intrigeri about 3 years ago

Branch seems to build now, but I will have to test if syslinux-i386 is really included and if we can upgrade.

Yeah! \o/

#15 Updated by u about 3 years ago

Setup: Boot on Tails 2.7. Connect Tails 2.7 USB. tails-installer -> upgrade from ISO. Choose Tails 3.0 with syslinux-i386 and amd64 versions installed.

2016-11-17 17:36:55,889 [creator.py:992 (install_bootloader)] INFO: Bootloader wird installiert …
2016-11-17 17:36:55,896 [creator.py:1314 (flush_buffers)] INFO: Daten auf dem Datenträger werden synchronisiert…
2016-11-17 17:36:55,974 [creator.py:793 (unmount_device)] INFO: Eingehängtes Dateisystem auf /dev/sda1 wird ausgehängt
2016-11-17 17:36:59,652 [creator.py:300 (popen)] INFO: sync

sync

/tmp/tmpPiER9m/syslinux  -d syslinux /dev/sda1
/bin/sh: 1: /tmp/tmpPiER9m/syslinux: not found

2016-11-17 17:36:59,653 [gui.py:259 (run)] ERROR: Es gab ein Problem beim Ausführen des folgenden Befehls »/tmp/tmpPiER9m/syslinux  -d syslinux /dev/sda1«.
Eine detailreichere Fehlerbeschreibung ist hier zu finden: »/tmp/tails-installer-r_UQED«
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 232, in run
    self.live.install_bootloader()
  File "/usr/lib/python2.7/dist-packages/tails_installer/creator.py", line 1053, in install_bootloader
    'syslinux', self.drive['device']))
  File "/usr/lib/python2.7/dist-packages/tails_installer/creator.py", line 303, in popen
    % {'command': cmd, 'filename': self._error_log_filename})
TailsInstallerError: Es gab ein Problem beim Ausf\xfchren des folgenden Befehls \xbb/tmp/tmpPiER9m/syslinux  -d syslinux /dev/sda1\xab.

But the file exists.

$ file utils/linux/syslinux*
utils/linux/syslinux:      ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=777388483f0fe9a43f271ca9a3ff05cf3b51845e, stripped
utils/linux/syslinux-i386: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=234287546e048776d41685fa36328d8ffe35ff7d, stripped

And we run on a 64bit kernel:

$ uname -a
Linux amnesia 4.7.0-0.bpo.1-amd64 #1 SMP Debian 4.7.8-1~bpo8+1 (2016-10-19) x86_64 GNU/Linux

It appears that the file is not executable.
Running manually `/mnt/utils/linux/syslinux-i386 -d syslinux /dev/sda1` instead of `/mnt/utils/linux/syslinux -d syslinux /dev/sda1` works.

So either we ship only the i386 binary of syslinux, because this is executable by both amd64 and i386, or we ship both and this will need a modification in tails-installer. tails-installer then needs to choose the correct syslinux depending ont the Tails we are currently running on.

What do you think?

#16 Updated by u about 3 years ago

Better idea: not modify the installer and just copy the 32-bit syslinux to the 64-bit executable in the hook.

#17 Updated by intrigeri about 3 years ago

Better idea: not modify the installer and just copy the 32-bit syslinux to the 64-bit executable in the hook.

Sounds good. Wanna do it?

#18 Updated by u about 3 years ago

I made a dirty fix for that and will test it again.

#19 Updated by u about 3 years ago

This would be the diff we would need to revert in Tails 3.3 in order to ship only syslinux amd64, instead of shipping only the 32-bit version.

diff --git a/config/binary_local-hooks/40-include_syslinux_in_ISO_filesystem b/config/binary_local-hooks/40-include_syslinux_in_ISO_filesystem
index cab4a3f..5d1ee95 100755
--- a/config/binary_local-hooks/40-include_syslinux_in_ISO_filesystem
+++ b/config/binary_local-hooks/40-include_syslinux_in_ISO_filesystem
@@ -21,6 +21,12 @@ Set_defaults
 # Seems like we'll have work to do
 Echo_message 'including syslinux in the ISO filesystem'

+### Functions
+
+syslinux_deb_version_in_chroot () {
+   chroot chroot dpkg-query -W -f='${Version}\n' syslinux
+}
+
 ### Variables
 LINUX_BINARY_UTILS_DIR='binary/utils/linux'
 WIN32_BINARY_UTILS_DIR='binary/utils/win32'
@@ -28,16 +34,36 @@ BINARY_MBR_DIR='binary/utils/mbr'
 CHROOT_SYSLINUX_BIN='chroot/usr/bin/syslinux'
 CHROOT_SYSLINUX_MBR='chroot/usr/lib/SYSLINUX/gptmbr.bin'
 CHROOT_TEMP_APT_SOURCES='chroot/etc/apt/sources.list.d/tmp-deb-src.list'
-
-### Functions
-
-syslinux_deb_version_in_chroot () {
-   chroot chroot dpkg-query -W -f='${Version}\n' syslinux
-}
+SYSLINUX_DEB_VERSION_IN_CHROOT=$(syslinux_deb_version_in_chroot)

 ### Main
 mkdir -p "$LINUX_BINARY_UTILS_DIR" "$WIN32_BINARY_UTILS_DIR" "$BINARY_MBR_DIR" 
-cp "$CHROOT_SYSLINUX_BIN" "$LINUX_BINARY_UTILS_DIR/" 
+# Copy 64-but syslinux binary
+# We only need the 32-bit binary until most of the users have upgraded to 64-bit
+# Copy 32-bit syslinux binary
+(
+   olddir=$(pwd)
+   workdir=$(mktemp -d)
+   cd "$workdir" 
+   chroot="$olddir/chroot" 
+   echo "Configuring APT architectures for the installation of syslinux" 
+   Chroot "$chroot" \
+      echo 'APT::Architectures {"i386"; "amd64";};' \
+      > /etc/apt/apt.conf.d/13architectures
+   Chroot "$chroot" dpkg --add-architecture amd64
+   Chroot "$chroot" apt-get update
+   echo "Trying to download syslinux-i386 version ${SYSLINUX_DEB_VERSION_IN_CHROOT}" 
+   Chroot "$chroot" \
+      apt-get --yes download \
+      syslinux:i386="${SYSLINUX_DEB_VERSION_IN_CHROOT}" 
+   echo "Extracting syslinux-i386" 
+   dpkg-deb --extract "$chroot"/syslinux_*.deb .
+   rm "$chroot"/syslinux_*.deb
+   cp ./usr/bin/syslinux "$olddir/$LINUX_BINARY_UTILS_DIR/" 
+   cd "$olddir" 
+   rm -r "$workdir" 
+)
+# Copy syslinux MBR
 cp "$CHROOT_SYSLINUX_MBR" "$BINARY_MBR_DIR/mbr.bin" 

 cat chroot/etc/apt/sources.list chroot/etc/apt/sources.list.d/*.list \

#20 Updated by u about 3 years ago

  • Assignee changed from u to intrigeri
  • QA Check set to Ready for QA

It works! yay! May you please check if you like the proposed solution?

#22 Updated by intrigeri about 3 years ago

  • Feature Branch changed from 451f:tails/bug/11873+upgrade_32-bit_to_64-bit_ISO to bug/11873+upgrade_32-bit_to_64-bit_ISO

#23 Updated by intrigeri about 3 years ago

Thanks! I've polished the branch a tiny bit.

I'm surprised that we need to dpkg --add-architecture amd64 (my intuition tells me that dpkg --add-architecture i386 might be needed as we're on an amd64 system, but I don't understand why we would need to explicitly enable amd64). This might work, and be actually needed, because of a problem in config/chroot_local-hooks/99-disable-multiarch so I'll fix #11961 first and then I'll come back here. I suspect that I'll have to s/amd64/i386/ in the aforementioned command, as a result.

#24 Updated by intrigeri about 3 years ago

  • Blocked by Bug #11961: Drop obsolete multiarch handling in Stretch added

#25 Updated by u about 3 years ago

intrigeri wrote:

Thanks! I've polished the branch a tiny bit.

<3

I'm surprised that we need to dpkg --add-architecture amd64 (my intuition tells me that dpkg --add-architecture i386 might be needed as we're on an amd64 system, but I don't understand why we would need to explicitly enable amd64). This might work, and be actually needed, because of a problem in config/chroot_local-hooks/99-disable-multiarch so I'll fix #11961 first and then I'll come back here. I suspect that I'll have to s/amd64/i386/ in the aforementioned command, as a result.

Indeed, that's a mistake! I wanted to add the i386 architecture. But - it worked, so actually, we might even be able to delete this line.

#26 Updated by intrigeri about 3 years ago

  • % Done changed from 10 to 50

With one last fix on top of the branch, it builds! Merged into feature/stretch. I'll now look at test suite results, and it would be nice if someone tested upgrading with the resulting ISO from Tails 2.x. u, do you want to do that?

#27 Updated by intrigeri about 3 years ago

Since I've merged this branch, the USB install & upgrade tests fail: https://jenkins.tails.boum.org/view/Tails_ISO/job/test_Tails_ISO_feature-stretch/33/. The extracted syslinux binary apparently cannot be run.

#28 Updated by intrigeri about 3 years ago

  • QA Check changed from Ready for QA to Dev Needed

#29 Updated by intrigeri almost 3 years ago

  • QA Check changed from Dev Needed to Ready for QA

febdcdc repaired it: https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_feature-stretch/lastFailedBuild/cucumberTestReport/

Next step is to try upgrading using a current feature/stretch ISO, from a 32-bit Tails, which was the point of this ticket initially.

#30 Updated by intrigeri almost 3 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

#31 Updated by intrigeri over 2 years ago

  • Related to Feature #13513: Replace syslinux:i386 with syslinux:amd64 in the ISO9660 filesystem added

#32 Updated by intrigeri over 2 years ago

Will revert this in #13513.

Also available in: Atom PDF