Project

General

Profile

Feature #11151

Speed up the build of the website

Added by sajolida over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
12/23/2015
Due date:
% Done:

100%

Feature Branch:
Type of work:
Website
Blueprint:
Starter:
Affected tool:

Description

As suggested in #11135#note-5 we could measure the time to build the wiki in the past (without the Installation Assistant for example) and now, and try to set a realistic goal.

Then we should investigate on concrete changes to speed things up, like #11135.


Subtasks

Feature #10792: Try to group inc/steps inlinesResolved

Bug #11135: Disable translations of /news/reportsResolved

Feature #11156: Disable translations about very old releasesResolved

Feature #11158: Disable backlinks on websiteRejected

Bug #11227: Website: Broken links at https://tails.boum.org/security/Resolvedsajolida

Feature #11249: Do stats on which pages are the least translatedResolved


Related issues

Related to Tails - Feature #9323: Adapt current documentation with the Installation Assistant Resolved 11/27/2015

History

#1 Updated by sajolida over 3 years ago

  • Tracker changed from Bug to Feature
  • Subject changed from Speed the build of the wiki to Speed the build of the website

#2 Updated by sajolida over 3 years ago

Reference time, on a X200 with a P8700, building from RAM on Tails takes at commit edf97d3:

real    48m53.368s
user    48m4.740s
sys    0m15.344s

#3 Updated by intrigeri over 3 years ago

On intri's laptop:

current

git reset --hard 89d07a0245d7f09707aefa1fa0ad8c82ab5e8ac7 && time ./build-website --rebuild

1036.49s user 6.94s system 100% cpu 17:17.74 total

old school (no Farsi, no IA)

git reset --hard caa53db1bf9f0157207ba92f56daf23f527ac6e3^ && time ./build-wiki --rebuild

334.42s user 3.51s system 99% cpu 5:38.40 total

current minus Farsi

git reset --hard 89d07a0245d7f09707aefa1fa0ad8c82ab5e8ac7
patch -p1<<EOF
--- a/ikiwiki.setup
+++ b/ikiwiki.setup
@@ -199,7 +199,6 @@ po_master_language: en|English
 # slave languages (translated via PO files) format: ll|Langname
 po_slave_languages:
   - de|Deutsch
-  - fa|Farsi
   - fr|Français
   - pt|Português
 # PageSpec controlling which pages are translatable
EOF
find wiki/src -name *.fa.po -delete
time ./build-website --rebuild

738.55s user 5.02s system 100% cpu 12:18.17 total

current minus IA

git reset --hard 89d07a0245d7f09707aefa1fa0ad8c82ab5e8ac7
rm -rf wiki/src/{install,upgrade}
time ./build-website --rebuild

604.88s user 5.70s system 99% cpu 10:10.96 total

Conclusions

no IA, no Farsi = about 5.5 minutes, and then IA alone adds 7 minutes, Farsi alone adds 5 minutes, and adding both together adds 12 minutes => almost 17.5 minutes

#4 Updated by sajolida over 3 years ago

From the same commit after doing rm -r install upgrade I get:

real    30m22.595s
user    29m6.884s
sys    0m11.628s

which is 39% less and matches more or less what intrigeri gets.

Let's try to reduce the build time by 1/3 then.

#5 Updated by sajolida over 3 years ago

After merging #11135 and #9671 I get:

real    43m19.589s
user    41m47.072s
sys    0m15.312s

which is 11% less already.

#6 Updated by sajolida over 3 years ago

Remove the old installation instructions (/install and /upgrade) also speeds about 4%.

#7 Updated by sajolida over 3 years ago

  • Related to Feature #9323: Adapt current documentation with the Installation Assistant added

#8 Updated by intrigeri over 3 years ago

At 9244ee23feb893aa1bf956b825bf0d0aa225e145 (after merging the branches for #11135, #9671 and #11156) I see:

779.86s user 4.99s system 101% cpu 12:56.60 total

i.e. -25% :)

#9 Updated by intrigeri over 3 years ago

  • Subject changed from Speed the build of the website to Speed up the build of the website
  • Status changed from Confirmed to In Progress

#10 Updated by anonym over 3 years ago

On my laptop (Intel i7-3520M CPU with the core used by ikiwiki running at 3.2 GHz, and a pretty fast SSD):

current

real    20m23.051s
user    20m39.604s
sys     0m7.780s

old school (no Farsi, no IA)

real    6m18.782s
user    6m20.312s
sys     0m3.664s

current minus Farsi

real    13m2.951s
user    13m19.244s
sys     0m5.444s

current minus IA

real    10m6.676s
user    10m8.244s
sys     0m4.436s

So very comparable figures to what you've all seen.

So, in my case the new target (- 1/3) would sill be 20m23s * 2/3 ~= 14m which is still pretty long. :/

#11 Updated by sajolida over 3 years ago

  • Related to Bug #11227: Website: Broken links at https://tails.boum.org/security/ added

#12 Updated by intrigeri over 3 years ago

  • Related to deleted (Bug #11227: Website: Broken links at https://tails.boum.org/security/)

#13 Updated by emmapeel over 3 years ago

In my Thinkpad the current website takes

real    9m18.094s
user    8m58.084s
sys    0m9.280s

I have 7.6 GiB memory, and an Intel® Core™ i5 CPU M 520 @ 2.40GHz × 4 processor

#14 Updated by sajolida over 3 years ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (sajolida)

When building #9323 I get 11% less in comparison with master today, for a total of 25m41.124s which is 47% less than what I had initially. So I can close this ticket :) \o/

#15 Updated by sajolida over 3 years ago

On an X201 with an i3 M350:

real   10m55.939s
user   10m57.772s
sys    0m5.136s

#16 Updated by sajolida over 3 years ago

Today I've got 36m20.637s on my machine. That's 26% less than my reference time.

Also available in: Atom PDF