Project

General

Profile

Feature #15190

Feature #10034: Translation web platform

Feature #11759: Install & configure a fresh weblate on the VM

Cronjobs for translate.lizard are not running

Added by emmapeel almost 2 years ago. Updated 4 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
01/17/2018
Due date:
% Done:

80%

Feature Branch:
emmapeel:puppet-tails/less_cronjobs
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:

Description

There are some cronjobs the weblate install needs to run:

emmapeel@translate:/usr/local/share/weblate$ sudo -u weblate ./manage.py loadpo --all

This one is likely to give errors because of files missing if some files were moved in tails master repo (explained at
#15185). Will load as translations all .po files it finds on the proper folders.

emmapeel@translate:/usr/local/share/weblate$ sudo -u weblate ./manage.py updatechecks --all

This one will perform again the checks of the 'Failing checks' section, very useful for long-term translation maintenance:

https://translate.tails.boum.org/checks/inconsistent/tails/?language=it

emmapeel@translate:/usr/local/share/weblate$ sudo -u weblate ./manage.py  rebuild_index --all

Will rebuild the search index.


Related issues

Related to Tails - Bug #15679: update puppet-tails to have scripts in weblate vm Resolved 06/20/2018
Related to Tails - Bug #15185: Automatically update Weblate components on mdwn file add/rename/delete in tails.bo git Resolved 02/27/2019

History

#1 Updated by emmapeel almost 2 years ago

  • Blocks Feature #15079: Integrate the platform with our Git and ikiwiki infrastructure added

#2 Updated by emmapeel over 1 year ago

  • % Done changed from 0 to 50
  • QA Check changed from Dev Needed to Info Needed

I have installed the cronjobs:

  1. cronjobs for weblate, see:
  2. https://docs.weblate.org/en/weblate-2.10.1/admin/install.html#production-cron
  3. Fulltext index updates
    /5 * * * * cd /usr/share/weblate/; ./manage.py update_index
    /6 * * * * cd /usr/share/weblate/; ./manage.py updatechecks --all
    */7 * * * * cd /usr/share/weblate/; ./manage.py loadpo --all
  4. Cleanup stale objects
    @daily cd /usr/share/weblate/; ./manage.py cleanuptrans
  5. Commit pending changes after 96 hours
    @hourly cd /usr/share/weblate/; ./manage.py commit_pending --all --age=96 --verbosity=0

with crontab -e as user weblate.

I wonder is there is any puppetinzaing thing to be done though (or we can close this ticket)

#3 Updated by u over 1 year ago

  • Assignee set to emmapeel

Hi emmapeel!

thanks for your work! May you please add to our documentation

  • where you added these cronjobs
  • for each of them describe what they do
  • for each of them what happens if they fail.

Thanks!

#4 Updated by emmapeel over 1 year ago

  • Assignee deleted (emmapeel)

They are in the documentation of course. Only problem is people will not read it.

#5 Updated by u over 1 year ago

  • Assignee set to emmapeel
  • Feature Branch set to translate-server: master

emmapeel wrote:

They are in the documentation of course. Only problem is people will not read it.

Uuh, there is absolutely nothing detailed in the documentation about these cronjobs.

The latest commit I see on this repo is dde4906b22c2be851458cac5e20a493da4ba20d9.
In which file is the detailed doc of the cronjobs supposed to be found? grep at least did not even find the word cron so either I miss some details here (is it on the master branch?) or you forgot to push some changes you made.

Basically, what is written here in the ticket description is missing from the Git repo.

#6 Updated by u over 1 year ago

  • QA Check deleted (Info Needed)

I'll add them myself for now.

#7 Updated by u over 1 year ago

u wrote:

I'll add them myself for now.

Done in #13286.

#8 Updated by emmapeel over 1 year ago

u wrote:

Uuh, there is absolutely nothing detailed in the documentation about these cronjobs.

Well, there was:

Refresh translations
--------------------

These commands are executed daily by cronjobs, so we don't have to
execute them manually (see https://labs.riseup.net/code/issues/15190):

- Reloads all translations from proper folder on disk (eg. in case you
did some updates in VCS repository) to components of weblate
sudo -u weblate ./manage.py loadpo --all

- Cleanup orphaned checks and translation suggestions:
sudo -u weblate ./manage.py cleanuptrans

I think you edited it adding the word 'cronjobs'. I would not call that absolutely nothing.

#9 Updated by emmapeel over 1 year ago

  • Assignee deleted (emmapeel)

I still want to know if there is more to be done regarding puppetization of this cronjobs, or we can close this ticket.

#10 Updated by u over 1 year ago

  • Assignee set to emmapeel

emmapeel wrote:

u wrote:

Uuh, there is absolutely nothing detailed in the documentation about these cronjobs.

I think you edited it adding the word 'cronjobs'. I would not call that absolutely nothing.

Yes, the two commands you noted in the documentation were not specified to be cronjobs and it was unclear if they have to be run manually. I also described better what these cronjobs do, and how often they are run. -> All of this information was missing. Furthermore, I added an entire section at the bottom of the file about the other two cronjobs which were mentioned nowhere else.

#11 Updated by u over 1 year ago

  • Assignee changed from emmapeel to groente
  • QA Check set to Info Needed

As for your remaining question, we should ask sysadmins if there is anything left to do, rather than merely unassigning this ticket I believe. @groente: I'm unsure about puppetizing cronjobs, is this something we should consider doing?

#12 Updated by emmapeel over 1 year ago

Actually I think this cronjobs are still not running.

Maybe we need to allow weblate user to run cronjobs or similar?

Maybe we can just move them to /etc/cron.hourly/ ?

#13 Updated by u over 1 year ago

  • Target version set to Tails_3.7

Hi groente, may you please check why the cronjobs are not running? Thanks!

#15 Updated by u over 1 year ago

  • Blocks deleted (Feature #15079: Integrate the platform with our Git and ikiwiki infrastructure)

#16 Updated by u over 1 year ago

  • Parent task set to #15079

#17 Updated by u over 1 year ago

  • Subject changed from Some cronjobs for translate.lizard to Cronjobs for translate.lizard are not running

#18 Updated by groente over 1 year ago

  • Assignee changed from groente to emmapeel
  • QA Check changed from Info Needed to Dev Needed

I added some checks to the cronjobs (writing to a tmp-file) and they are definately running. Perhaps the commands are missing some environment variables to function properly? Either way, please keep them in regular crontab and don't move them to cron.daily.

As for puppetising, that would be preferrable, I can do that as soon as the commands actually produce the desired output (please reassign this ticket to me once this is the case).

#19 Updated by emmapeel over 1 year ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from emmapeel to groente

Ok, please puppetize them then!

#20 Updated by u over 1 year ago

  • QA Check deleted (Dev Needed)

emmapeel wrote:

Ok, please puppetize them then!

@emmapeel: If you reassign this to groente, this means these commands work correctly? He mentioned there might be missing variables. Why did you think they were not running in first place? Can you please make this clear on the ticket? Thanks.

And then, yes, let's puppetize it.

#21 Updated by emmapeel over 1 year ago

  • QA Check set to Dev Needed

Please puppetize the commands that are running in the weblate user contrab. Maybe they were blocked by ther processes before and that is why they were not running.

There have been some merges on the repo but it is hard to know if they were triggered by this crontab or not.

#22 Updated by groente over 1 year ago

  • Assignee changed from groente to emmapeel
  • QA Check changed from Dev Needed to Ready for QA

the cronjobs are now puppetised, can you confirm they are working properly?

#23 Updated by bertagaz over 1 year ago

  • Target version changed from Tails_3.7 to Tails_3.8

#24 Updated by emmapeel over 1 year ago

  • Status changed from In Progress to Resolved
  • QA Check changed from Ready for QA to Pass

Yes, they look good!

#25 Updated by emmapeel over 1 year ago

  • Status changed from Resolved to In Progress
  • Assignee changed from emmapeel to groente
  • QA Check changed from Pass to Ready for QA
  • Feature Branch changed from translate-server: master to emmapeel/puppet-tails:better_dir

As this tasks are also trigered by other processes, it took me a while to realize the cronjobs have to point to another directory.

Please review this branch:

git.tails.boum.org:emmapeel/puppet-tails.git * [new branch] better_dir -> better_dir

With the correct directory

#26 Updated by groente over 1 year ago

  • Status changed from In Progress to Resolved
  • Assignee changed from groente to emmapeel
  • QA Check changed from Ready for QA to Pass

I didn't have read access to your repo, but changed the directory in the cronjobs to what I believe is now the right one. Please test and reopen this ticket if there's still something wrong with the cronjobs.

#27 Updated by emmapeel over 1 year ago

Yes, it worked! Unfortunately, the cronjobs apparently are too much for the resources of the server and we are running out of memory quite often now.

I would not mind you having access to my repo, but i think it is public:

https://git-tails.immerda.ch/emmapeel/puppet-tails/

The cronjobs are working, but maybe they are too much, the server is being overloaded often.

I changed a little the frequency of them with 755fa48e9b978569c4bc215b144647e7515cfc45

I attach a patch here for you to review, otherwise is the branch less_cronjobs in said repository.

Please review

#28 Updated by emmapeel over 1 year ago

  • Related to Bug #15679: update puppet-tails to have scripts in weblate vm added

#29 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_3.8 to Tails_3.9

#30 Updated by groente over 1 year ago

  • % Done changed from 50 to 80
  • QA Check changed from Ready for QA to Pass

the job that was causing problems was the updatechecks, i've now set that to hourly instead of every six minutes. let's see how the machine holds now.

#31 Updated by emmapeel over 1 year ago

ey there, so there is a missing option on the cronjob at https://git-tails.immerda.ch/puppet-tails/tree/manifests/weblate.pp#n223 :

It should be "./manage.py updategit --all", not "./manage.py updategit".

(Also, I think this git update will be taken care of by new scripts, and maybe it can be run once an hour too... I don't think we need to bother other repos so often). Would you like a patch for this?

#32 Updated by groente over 1 year ago

  • Assignee changed from groente to emmapeel
  • QA Check changed from Pass to Ready for QA

I changed the cronjob, is everything working okay now?

#33 Updated by emmapeel over 1 year ago

  • Assignee changed from emmapeel to u

The changes are OK, now the cronjob is working correctly.

On the other hand, I assign this ticket to u to

- see if it is really necessary to update git with this cronjob, or we should drop it in exchange for hefee's new scripts ( #15185 )
- if we drop it, when? (it seems to me it will be replaced when #15185 is solved)

#34 Updated by emmapeel over 1 year ago

  • Related to Bug #15185: Automatically update Weblate components on mdwn file add/rename/delete in tails.bo git added

#35 Updated by u about 1 year ago

  • Target version changed from Tails_3.9 to Tails_3.10.1

emmapeel wrote:

The changes are OK, now the cronjob is working correctly.

On the other hand, I assign this ticket to u to

- see if it is really necessary to update git with this cronjob, or we should drop it in exchange for hefee's new scripts ( #15185 )
- if we drop it, when? (it seems to me it will be replaced when #15185 is solved)

Let's talk about it at the summit.

#36 Updated by u about 1 year ago

  • QA Check deleted (Ready for QA)

#37 Updated by u about 1 year ago

  • Status changed from In Progress to Resolved

We said that we want to schedule some work time together with a sysadmin to look into this.
Now, it seems to me that the cronjobs are running now. And yes, we will replace them with hefee's script and hence I think #1⁵190 is obsolete → closing.

#38 Updated by bertagaz about 1 year ago

  • Status changed from Resolved to In Progress
  • Assignee changed from u to emmapeel
  • QA Check set to Info Needed

Hi,

I'm loosely repopening this ticket to notice you that I've fixed some cronjob that were badly configured (see #15906#note-4 for more). Just to be sure I did not mess up anything, would you mind having a look at https://git-tails.immerda.ch/puppet-tails/tree/manifests/weblate.pp#n178 and the translate system/website? If it's ok feel free to set this ticket back in its previous state (status: resolved, assignee: u). I know you'll quit the cronjob implementation, but meanwhile I just want to be sure I did not break something. Thanks!

#39 Updated by intrigeri 12 months ago

  • Target version changed from Tails_3.10.1 to Tails_3.11

#40 Updated by hefee 12 months ago

  • Status changed from In Progress to Resolved

the cron jobs are now done via one file, so they can't lock themselves.

#41 Updated by u 8 months ago

  • Parent task changed from #15079 to #11759

#42 Updated by intrigeri 4 months ago

  • Assignee deleted (emmapeel)

Also available in: Atom PDF