Project

General

Profile

Bug #16637

squashfs-tools from Buster (1:4.3-12) is much slower than 1:4.3-8.0tails1

Added by intrigeri 5 months ago. Updated 19 days ago.

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Build system
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
bugfix/16637-faster-mksquashfs
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Compressing the SquashFS on lizard jumped from 20 min to 38 min since we upgraded from 1:4.3-8.0tails1.


Related issues

Related to Tails - Bug #16457: Building feature/buster in RAM sometimes fails: mksquashfs reaped by OOM killer Resolved 02/11/2019
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision bd933129 (diff)
Added by segfault 4 months ago

Enable localization for new locales introduced in Tor Browser 8.5 (refs: #16637)

Revision cdf270b4 (diff)
Added by intrigeri 22 days ago

Use squashfs-tools from sid (refs: #16637)

Revision 6d5fc2a9
Added by segfault 20 days ago

Merge branch 'bugfix/16637-faster-mksquashfs' into devel (Closes: #16637)

History

#1 Updated by intrigeri 5 months ago

  • Related to Bug #16457: Building feature/buster in RAM sometimes fails: mksquashfs reaped by OOM killer added

#2 Updated by intrigeri 5 months ago

@lamby, this one might be interesting for you.

#3 Updated by intrigeri 5 months ago

#4 Updated by segfault 4 months ago

  • Status changed from Confirmed to In Progress

#5 Updated by intrigeri 4 months ago

  • Status changed from In Progress to Confirmed

(That commit is unrelated, typo for #16337)

#6 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#7 Updated by intrigeri 22 days ago

  • Assignee set to intrigeri
  • Priority changed from Normal to Elevated

I'm tentatively making this a release blocker because of the combination of two factors:

  • Having our CI builds take about 30% more time has a painful impact on developer experience: our feedback loop is way too long already and there are not so many low-hanging fruits that would save 20 minutes.
  • This problem seems to be https://bugs.debian.org/921146, which is fixed in sid, so it might be that this is cheap to fix. I'll give it a try.

#8 Updated by intrigeri 22 days ago

  • Feature Branch set to bugfix/16637-faster-mksquashfs

#9 Updated by intrigeri 22 days ago

  • Status changed from Confirmed to Needs Validation

(Forcing reproducibility tests on CI.)

#10 Updated by intrigeri 22 days ago

  • Status changed from Needs Validation to In Progress

#11 Updated by intrigeri 22 days ago

  • Status changed from In Progress to Needs Validation

It works: on isobuilder1.sid, build goes down from ~50 min to ~40 min.

#12 Updated by intrigeri 22 days ago

First build on lizard took 65min, that is only slightly more than stable, and much less than devel, so I'm calling this a success. I'm trying to sneak another thing on top of this branch before sending it to QA.

#13 Updated by intrigeri 22 days ago

  • Assignee deleted (intrigeri)

Regarding c448a322b7784b7b0fa428197a5cdc218753b75c, it worked fine so far on lizard, my local Jenkins, and my own laptop (both with "ram" , "noram", "defaultcomp noram", and "defaultcomp ram" build options).

But none of these use the internal apt-cacher-ng, and I'd rather not download GBs of data just to test this, so I'd appreciate if the reviewer used that one and tried the "ram defaultcomp" and "noram" build options (I'm assuming that it'll give us enough coverage and one does not need to test each cell of the 2×2 matrix). Thanks in advance!

#14 Updated by segfault 21 days ago

  • Assignee set to segfault

#15 Updated by segfault 21 days ago

LGTM, will now test with apt-cacher-ng and "ram defaultcomp" and "noram" build options

#16 Updated by segfault 20 days ago

Both builds were fine.

#17 Updated by segfault 20 days ago

  • Status changed from Needs Validation to Resolved
  • % Done changed from 0 to 100

#18 Updated by intrigeri 19 days ago

Yeepee! :)

Also available in: Atom PDF