Project

General

Profile

Feature #7706

Feature #7705: Better compress the SquashFS

Better compress the SquashFS in the ISO

Added by intrigeri almost 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Build system
Target version:
Start date:
07/31/2014
Due date:
% Done:

100%

Feature Branch:
feature/7705-smaller-squashfs
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Related issues

Related to Tails - Bug #7462: isohybrid command returns warnings on Tails 1.1 Resolved 07/31/2014
Related to Tails - Bug #6372: Fix SquashFS sort file update slows down boot Resolved 10/17/2013

Associated revisions

Revision 41a6531d (diff)
Added by Tails developers almost 5 years ago

Compress the SquashFS more aggressively (Closes: #7706).

In my tests on current experimental branch, this makes the ISO 10% smaller, but
makes the build take 63% more (real) time, and eat 110% more (user) CPU time.
It seems worth it for released ISO images.

Likely, most developers that didn't do it yet will now set
MKSQUASHFS_OPTIONS='-comp gzip' when doing their regular builds... although the
time gained by gzip (vs. aggressive xz) at build time is compensated, in many
cases, by faster IO (e.g. when copying the ISO out of the build VM, when booting
the ISO, when installing the ISO on a USB stick).

History

#1 Updated by intrigeri almost 5 years ago

  • Feature Branch set to feature/7705-smaller-squashfs

#2 Updated by intrigeri almost 5 years ago

  • Related to Bug #7462: isohybrid command returns warnings on Tails 1.1 added

#3 Updated by intrigeri almost 5 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 50

Implemented. In my tests on current experimental branch, this makes the ISO 10% smaller, but makes the build take 63% more (real) time, and eat 110% more (user) CPU time. It seems worth it for released ISO images.

Likely, most developers that didn't do it yet will now set MKSQUASHFS_OPTIONS='-comp gzip' when doing their regular builds... although the time gained by gzip (vs. aggressive xz) at build time is compensated, in many cases, by faster IO (e.g. when copying the ISO out of the build VM, when booting the ISO, when installing the ISO on a USB stick).

I'll request a review'n'merge once the same trick is implemented for IUK's (#7707).

#4 Updated by intrigeri almost 5 years ago

An ISO built from the experimental branch (that has this one merged in) passes the automated test suite.

#5 Updated by intrigeri almost 5 years ago

  • Assignee changed from intrigeri to alant
  • Target version changed from Tails_1.2 to Tails_1.1.1
  • QA Check set to Ready for QA

Nominating for 1.1.1, as it seems to be the shorter way we have to fix the many boot problems caused by #7462.

#6 Updated by intrigeri almost 5 years ago

  • Assignee changed from alant to intrigeri
  • QA Check changed from Ready for QA to Info Needed

sajolida reported a doubled (!) boot time on the experimental branch, that might be related to this one. Just sent a call for testing, so hold on a bit.

#7 Updated by intrigeri almost 5 years ago

  • Related to Bug #6372: Fix SquashFS sort file update slows down boot added

#8 Updated by intrigeri almost 5 years ago

  • Assignee changed from intrigeri to sajolida

I think this might be resolved. Requested additional tests on tails-dev@.

#9 Updated by intrigeri almost 5 years ago

I've been sent these results, from the kernel loading to a fully loaded desktop:

  • VirtualBox, 1 core, 512MB RAM, 1.1: 50 seconds
  • VirtualBox, 1 core, 512MB RAM, experimental: 51 seconds
  • KVM, 1 core, 512MB RAM, 1.1: 40 seconds
  • KVM, 1 core, 512MB RAM, experimental: 49 seconds
  • Virtualbox 2 cores, 1536MB RAM, 1.1: 50 seconds
  • Virtualbox 2 cores, 1536MB RAM, experimental: 50 seconds
  • KVM, 2 cores, 1536MB RAM, 1.1: 37 seconds
  • KVM, 2 cores, 1536MB RAM, experimental: 38 seconds

The experimental ISO being used was tails-i386-experimental-1.2-20140812T0731Z-7554dc0.iso.

#10 Updated by intrigeri almost 5 years ago

Here are more test results. All times are from "pressing Enter in the bootloader" to "fully loaded Greeter".

libvirt/QEMU VM, booting from bare metal DVD:

  • 1.1: 1:38
  • stable branch with reordered SquashFS: 0:48
  • experimental with smaller but badly ordered SquashFS (similar to what sajolida tested, assuming he was not testing a gzip compressed SquashFS): 1:39
  • experimental with smaller and reordered SquashFS sort file: 0:54

MacBook Pro 8,1, booting from USB:

  • 1.1: 0:36
  • experimental with smaller but badly ordered SquashFS (20140809): 0:49
  • experimental with smaller and reordered SquashFS (20140813): 0:35

ThinkPad X32, booting from USB:

  • 1.1: 1:03
  • experimental with smaller but badly ordered SquashFS (20140809): 1:30
  • experimental with smaller and reordered SquashFS (20140813): 1:08

ThinkPad X60, booting from USB:

  • 1.1: 0:48
  • experimental with smaller but badly ordered SquashFS (20140809): 1:14
  • experimental with smaller and reordered SquashFS (20140813): 0:52

Compaq 615, booting from USB:

  • 1.1: 1:03
  • experimental with smaller and reordered SquashFS: 1:16

#11 Updated by intrigeri almost 5 years ago

  • Assignee deleted (sajolida)
  • QA Check changed from Info Needed to Ready for QA

As explained on tails-dev@, given the above results, I think this can safely go into 1.1.1, if only to workaround #7462.

#12 Updated by alant almost 5 years ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

Merged, thanks.

#13 Updated by anonym almost 5 years ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF