Project

General

Profile

Bug #16637

squashfs-tools from Buster (1:4.3-12) is much slower than 1:4.3-8.0tails1

Added by intrigeri 10 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Build system
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
bugfix/16637-faster-mksquashfs
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Compressing the SquashFS on lizard jumped from 20 min to 38 min since we upgraded from 1:4.3-8.0tails1.


Related issues

Related to Tails - Bug #16457: Building feature/buster in RAM sometimes fails: mksquashfs reaped by OOM killer Resolved 02/11/2019
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision bd933129 (diff)
Added by segfault 9 months ago

Enable localization for new locales introduced in Tor Browser 8.5 (refs: #16637)

Revision cdf270b4 (diff)
Added by intrigeri 5 months ago

Use squashfs-tools from sid (refs: #16637)

Revision 6d5fc2a9
Added by segfault 5 months ago

Merge branch 'bugfix/16637-faster-mksquashfs' into devel (Closes: #16637)

History

#1 Updated by intrigeri 10 months ago

  • Related to Bug #16457: Building feature/buster in RAM sometimes fails: mksquashfs reaped by OOM killer added

#2 Updated by intrigeri 10 months ago

@lamby, this one might be interesting for you.

#3 Updated by intrigeri 10 months ago

#4 Updated by segfault 9 months ago

  • Status changed from Confirmed to In Progress

#5 Updated by intrigeri 9 months ago

  • Status changed from In Progress to Confirmed

(That commit is unrelated, typo for #16337)

#6 Updated by intrigeri 6 months ago

  • Description updated (diff)

#7 Updated by intrigeri 5 months ago

  • Assignee set to intrigeri
  • Priority changed from Normal to Elevated

I'm tentatively making this a release blocker because of the combination of two factors:

  • Having our CI builds take about 30% more time has a painful impact on developer experience: our feedback loop is way too long already and there are not so many low-hanging fruits that would save 20 minutes.
  • This problem seems to be https://bugs.debian.org/921146, which is fixed in sid, so it might be that this is cheap to fix. I'll give it a try.

#8 Updated by intrigeri 5 months ago

  • Feature Branch set to bugfix/16637-faster-mksquashfs

#9 Updated by intrigeri 5 months ago

  • Status changed from Confirmed to Needs Validation

(Forcing reproducibility tests on CI.)

#10 Updated by intrigeri 5 months ago

  • Status changed from Needs Validation to In Progress

#11 Updated by intrigeri 5 months ago

  • Status changed from In Progress to Needs Validation

It works: on isobuilder1.sid, build goes down from ~50 min to ~40 min.

#12 Updated by intrigeri 5 months ago

First build on lizard took 65min, that is only slightly more than stable, and much less than devel, so I'm calling this a success. I'm trying to sneak another thing on top of this branch before sending it to QA.

#13 Updated by intrigeri 5 months ago

  • Assignee deleted (intrigeri)

Regarding c448a322b7784b7b0fa428197a5cdc218753b75c, it worked fine so far on lizard, my local Jenkins, and my own laptop (both with "ram" , "noram", "defaultcomp noram", and "defaultcomp ram" build options).

But none of these use the internal apt-cacher-ng, and I'd rather not download GBs of data just to test this, so I'd appreciate if the reviewer used that one and tried the "ram defaultcomp" and "noram" build options (I'm assuming that it'll give us enough coverage and one does not need to test each cell of the 2×2 matrix). Thanks in advance!

#14 Updated by segfault 5 months ago

  • Assignee set to segfault

#15 Updated by segfault 5 months ago

LGTM, will now test with apt-cacher-ng and "ram defaultcomp" and "noram" build options

#16 Updated by segfault 5 months ago

Both builds were fine.

#17 Updated by segfault 5 months ago

  • Status changed from Needs Validation to Resolved
  • % Done changed from 0 to 100

#18 Updated by intrigeri 5 months ago

Yeepee! :)

Also available in: Atom PDF