Project

General

Profile

Feature #11113

Feature #11009: Improve ISO building and testing throughput and latency

Run more ISO testers

Added by intrigeri over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
Continuous Integration
Target version:
Start date:
02/12/2016
Due date:
% Done:

100%

Starter:
Affected tool:

Description

Now that we have more RAM to do so..


Related issues

Related to Tails - Feature #6729: Bump the number of CPU cores the testing VM has Resolved 02/19/2014
Blocks Tails - Feature #11011: Design a 2nd ISO testing machine Rejected 01/27/2016
Blocked by Tails - Feature #11010: Give lizard v2 more RAM Resolved 01/27/2016

History

#1 Updated by intrigeri over 3 years ago

#2 Updated by intrigeri over 3 years ago

#3 Updated by intrigeri over 3 years ago

And it'll be time to try running all isotesters with enough RAM to store TailsToaster's memory + the tmpfs peak usage + whatever we want to keep in disk cache e.g. the 2 ISO images + some memory for the system, that is ~23GB last time I checked (#10996#note-5).

Also see #10503#note-20 and follow-ups.

#4 Updated by intrigeri over 3 years ago

anonym: FYI I re-run a full test suite and I got 12688900 max usage in /tmp/TailsToaster again (run du -cs /tmp/TailsToaster/ | tail -n 1 | awk '{print $1}' in a loop, saving its output to a file, and then finding the maximum value in this file. I suspect that your measurements on #10503#note-20 were including a too small subset of what takes space in there.

#5 Updated by intrigeri over 3 years ago

Bumped isotester 1..6 to 23GiB of RAM (24117248 KiB).

#6 Updated by intrigeri over 3 years ago

  • Related to Feature #6729: Bump the number of CPU cores the testing VM has added

#7 Updated by intrigeri over 3 years ago

  • Blocked by Bug #10068: Upgrade to Jenkins 2.x, using upstream packages added

#8 Updated by intrigeri over 3 years ago

  • Blocked by deleted (Bug #10068: Upgrade to Jenkins 2.x, using upstream packages)

#9 Updated by intrigeri over 3 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 20

Set up isotester7 and 8, started benchmarking process.

#11 Updated by intrigeri over 3 years ago

  • Blueprint changed from https://tails.boum.org/blueprint/hardware_for_automated_tests_take2/ to https://tails.boum.org/blueprint/hardware_for_automated_tests_take2/#benchmarks

#12 Updated by intrigeri over 3 years ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 20 to 100

To sum up my benchmark results: running 8 ISO testers + giving them more RAM + making them use tmpfs for their temporary test suite data => we've improved our test suite runs throughput from 3.3 to 8 runs / hour. There's probably some more room for future improvements (#6729, #11175, #11179) but this part of the experiment is done, and is a great success.

Also available in: Atom PDF