Project

General

Profile

Feature #9788

Take benefit from squashfs-tools 4.3 for faster test builds

Added by romeopapa over 4 years ago. Updated 10 months ago.

Status:
Resolved
Priority:
Low
Assignee:
-
Category:
Build system
Target version:
Start date:
07/20/2015
Due date:
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Initial discussion and benchmarks: https://mailman.boum.org/pipermail/tails-dev/2015-July/009241.html

squashfs-tools 4.3 allows setting the gzip compression level. This means faster test builds.

Once our Vagrant basebox is based on Stretch, we can modify the gzipcomp build option to use a lower gzip compression level.


Related issues

Related to Tails - Bug #16177: Building ISO in RAM often fails: mksquashfs reaped by OOM killer Resolved 12/02/2018

Associated revisions

Revision 729235f6 (diff)
Added by intrigeri about 3 years ago

Use the lowest compression level for the SquashFS when compressing it with gzip.

According to https://mailman.boum.org/pipermail/tails-dev/2015-July/009241.html this
will make our development builds faster, and the resulting ISO image
a little bit bigger.

Closes: #9788

History

#1 Updated by romeopapa over 4 years ago

Ok I've submitted a big debdiff for squashfs-tools that updates to the latest source code, updates the manual pages and a patch that fixes a couple of opened CVEs. Hopefully it should be quick to get into sid!

#2 Updated by romeopapa over 4 years ago

Turns out I didn't send the bug report correctly, so nothing happened: Don't send an email to the bug submission with a 300K file attached, it'll just be ignored.

Here are the 3 bugs listed at the debian's bug tracker:

CVE-2015-4645:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793467

CVE-2015-4646:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793468

Update to 4.3+20140919 and man pages update:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793468

It's a little bit messy, but now they have everything they need to update the package!

#3 Updated by romeopapa over 4 years ago

Update to 4.3+20140919 and man pages update:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793468

Update to 4.3+20140919 and man pages update:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793480

Little typo fix.

#4 Updated by intrigeri over 4 years ago

  • Status changed from New to In Progress
  • Assignee set to romeopapa
  • % Done changed from 0 to 10

#5 Updated by intrigeri over 4 years ago

  • Subject changed from squashfs-tool 4.3 (faster test builds) to Use squashfs-tool 4.3 for faster test builds

#6 Updated by intrigeri about 4 years ago

4.3 is now in testing/sid, and could in turn be backported for Jessie.

#7 Updated by intrigeri over 3 years ago

  • Subject changed from Use squashfs-tool 4.3 for faster test builds to Use squashfs-tools 4.3 for faster test builds

Any progress on this front? Personally, I'm tempted to just wait until we switch our build system to Stretch and get this newer mksquashfs' features for free, but if someone feels like doing the work to get it earlier, well, great :)

#8 Updated by romeopapa over 3 years ago

Hey I'm back I'm gonna check out the status of the upstream patches i had submitted.

#9 Updated by intrigeri over 3 years ago

Hey I'm back I'm gonna check out the status of the upstream patches i had submitted.

Great!

#10 Updated by intrigeri over 3 years ago

  • Description updated (diff)

#11 Updated by intrigeri over 3 years ago

So, 4.3 is now in testing, great! So we'll have this for free when we migrate to Stretch (and it would be nice if someone tried this, e.g. when building our feature/stretch branch, on more powerful hardware), so this can become an easy goal for Tails 3.0. Now, if someone is excited about it and wants it earlier, then they need to have the package into jessie-backports. romeopapa, what are your plans?

#12 Updated by intrigeri over 3 years ago

  • Subject changed from Use squashfs-tools 4.3 for faster test builds to Take benefit from squashfs-tools 4.3 for faster test builds
  • Description updated (diff)
  • Assignee deleted (romeopapa)
  • Priority changed from Normal to Low
  • Target version set to Tails_3.0
  • Type of work changed from Debian to Code

intrigeri wrote:

So, 4.3 is now in testing, great! So we'll have this for free when we migrate to Stretch (and it would be nice if someone tried this, e.g. when building our feature/stretch branch, on more powerful hardware), so this can become an easy goal for Tails 3.0. Now, if someone is excited about it and wants it earlier, then they need to have the package into jessie-backports. romeopapa, what are your plans?

Time has passed so I'm adjusting this ticket's description to make it clear it's about Stretch.

#13 Updated by intrigeri about 3 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 10 to 100

#14 Updated by intrigeri 10 months ago

  • Related to Bug #16177: Building ISO in RAM often fails: mksquashfs reaped by OOM killer added

#15 Updated by intrigeri 10 months ago

FTR this upgrade causes #16177, because mksquashfs will now use by default 25% of the physical memory, which is way too much when most of that memory is used by the tmpfs that backs the build directory.

Lesson learnt: next time, read the changelog :)

Also available in: Atom PDF