Project

General

Profile

Feature #15360

Refresh the Weblate staging website more often

Added by u over 1 year ago. Updated 3 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
-
Category:
Infrastructure
Target version:
-
Start date:
03/02/2018
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:
Translation Platform

Description

- find out how we could do it
- if it would be possible to do it only for one file in one language

currently, the build with all languages enabled takes ~25 minutes, so it's much too long to ask for a rebuild of the entire wiki.

This is not part of the deliverable, but we would want that.


Related issues

Related to Tails - Feature #16872: Make it obvious to translators that they are visiting the staging website Confirmed
Blocked by Tails - Feature #15080: Integrate the staging website with the interface Confirmed 06/11/2018

History

#1 Updated by u over 1 year ago

  • Parent task deleted (#15077)

I think this is out of scope for the grant and we can do that as a later improvement to the staging website.

#2 Updated by u over 1 year ago

  • Parent task set to #10034

#3 Updated by u over 1 year ago

  • Parent task deleted (#10034)

#4 Updated by u over 1 year ago

#5 Updated by u over 1 year ago

Our current idea is to build the staging wiki once a day and I think this is sufficient for now.

#6 Updated by u 11 months ago

  • Assignee changed from u to emmapeel

hi emma, if you would like to look deeper into this, we could collect ideas on this ticket. But building once a day currently seems sufficient to me.

#7 Updated by hefee 10 months ago

The the main website using the ikiwiki pingee plugin to trigger a rebuild:
https://git.tails.boum.org/puppet-tails/tree/files/gitolite/hooks/www_website_ping-post-update.hook

#8 Updated by intrigeri 3 months ago

  • Affected tool set to Translation Platform

#9 Updated by intrigeri 3 months ago

#10 Updated by intrigeri 3 months ago

  • Related to Feature #15080: Integrate the staging website with the interface added

#11 Updated by intrigeri 3 months ago

  • Subject changed from Check for mechanisms that (safely) trigger staging builds accessible to all translators to Refresh the Weblate staging website more often
  • Category set to Infrastructure
  • Assignee deleted (emmapeel)
  • Type of work changed from Research to Sysadmin

We discussed this with @hefee and our current take on it:

  • Measure how long it takes to refresh (not rebuild) the staging website in common (not worst case) situations.
  • Assuming this refresh time is expressed in minutes rather than hours, run it as often as we can (without monopolizing a vCPU permanently), using a systemd timer (which gives us not only cron-like scheduling but also avoids running 2 instances of the refresh job at the same time).

In any case, this goes hand in hand with #15080 (either we do both or we do none of them):

  • If the staging website for the page one is reviewing is not readily available from the Weblate interface, then chances are translators don't use this staging website much (if at all).
  • If the staging website is lagging most of the time behind the changes one is reviewing, chances are that translators get used to ignore it because it's not useful.

#12 Updated by intrigeri 3 months ago

We currently don't refresh, we always rebuild.

I've tried --refresh and that fails because save-suggestions.py does git clean -fdx, which deletes the files ikiwiki needs to know what to refresh, namely config/chroot_local-includes/usr/share/doc/tails/website/.ikiwiki/. Would git clean -fd suffice?

#13 Updated by hefee 3 months ago

  • Assignee set to intrigeri

intrigeri wrote:

I've tried --refresh and that fails because save-suggestions.py does git clean -fdx, which deletes the files ikiwiki needs to know what to refresh, namely config/chroot_local-includes/usr/share/doc/tails/website/.ikiwiki/. Would git clean -fd suffice?

I think that should be suffice. Go for it.

#14 Updated by intrigeri 3 months ago

  • Status changed from Confirmed to In Progress
  • Target version set to Tails_3.16

I think that should be suffice. Go for it.

Done in https://git.tails.boum.org/puppet-tails/commit/?id=1bfb6238b172ac33cdb6f6592dc4b72ce934a127. We'll see how it goes tomorrow morning.

@hefee, please read the commit message, especially the part about POT files, and let me know if you can already predict trouble :)

#15 Updated by intrigeri 3 months ago

  • Assignee deleted (intrigeri)
  • Target version deleted (Tails_3.16)

intrigeri wrote:

  • Measure how long it takes to refresh (not rebuild) the staging website in common (not worst case) situations.

It takes about 1 hour:

weblate@translate:~$ time /var/lib/weblate/scripts/suggestions-cronjob.sh

real    60m18.435s
user    52m40.796s
sys     0m15.492s
  • Assuming this refresh time is expressed in minutes rather than hours, run it as often as we can (without monopolizing a vCPU permanently), using a systemd timer (which gives us not only cron-like scheduling but also avoids running 2 instances of the refresh job at the same time).

I think it would be reasonable to run this every 4 hours.

I'm done with this initial analysis and will come back to it once we've discussed with our team how we prioritize this + #15080.

#16 Updated by intrigeri 2 days ago

  • Related to Feature #16872: Make it obvious to translators that they are visiting the staging website added

#17 Updated by intrigeri 2 days ago

  • Related to deleted (Feature #15080: Integrate the staging website with the interface)

#18 Updated by intrigeri 2 days ago

  • Blocked by Feature #15080: Integrate the staging website with the interface added

Also available in: Atom PDF