Project

General

Profile

Bug #17347

[Weblate] get Celery working

Added by hefee about 1 month ago. Updated 23 days ago.

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:
Translation Platform

Description

/usr/local/weblate$ python3 manage.py check --deploy
SystemCheckError: System check identified some issues:

CRITICALS:
?: (weblate.E020) The Celery is not configured to store results, CELERY_RESULT_BACKEND is probably not set.
        HINT: https://docs.weblate.org/en/weblate-3.5.1/admin/install.html#celery

Weblate itself tells us that something is not correct with Celery ;D

At least we don't seen anything committed the last 24h, where stuff was modified! That's why we expect that celery is not working.
But also the knowledge how to get statistics or information out of celery is missing, to check if the others tasks are done by celery.

It seems that we need to configure a CELERY_RESULT_BACKEND - Celery upstream suggests to use a proper broker

At least for me I don't know what a broker is or why we need a result_backend. So either we find someone who can give us this information or we need to dig into that ourselves.


Related issues

Related to Tails - Bug #17370: [Weblate] Get registration working Resolved
Related to Tails - Bug #17340: Weblate: Check Celery settings for sane values Confirmed

History

#1 Updated by hefee about 1 month ago

  • Priority changed from Normal to Elevated
  • Target version set to Tails_4.2

#2 Updated by drebs 26 days ago

Hi, my understanding is that there are 2 different configs, and the one currently preventing celery to work is CELERY_BROKER_URL rather than CELERY_RESULT_BACKEND.

Regarding the configuration of a broker:

  • Celery is a distributed task queue.
  • Tasks are messages, so it needs something called a message broker to transport tasks from producers to consumers.
  • In the context of celery brokers can be either rabbitmq, redis or amazonsqs (there are also unmaintained experimental brokers, but i would probably rule them out unless we have more info on longer term maintainability).
  • We are currently using an in-memory broker, which is only good for testing because memory of different celery worker processes is not shared.
  • If we want to use celery, we need to install either redis or rabbitmq to use as a broker, and change this config appropriatelly.

Regarding the results backend, my understanding is:

So my conclusion is that we need to either install something like Redis to use as a broker or give up celery completelly. I don't currently have enough info to give a strong opinion on what's best, but I have a feeling that if Weblate is opting to use Celery as it's scheduling mechanism, then it might be easier to just fulfil this requirement than maintaining our own sets of cron jobs in the longer term.

Anything against using Redis as a message broker to make celery work?

#3 Updated by zen 26 days ago

  • Related to Bug #17370: [Weblate] Get registration working added

#4 Updated by hefee 26 days ago

  • Assignee set to drebs

drebs wrote:

Thanks for digging into Celery.

So my conclusion is that we need to either install something like Redis to use as a broker or give up celery completelly. I don't currently have enough info to give a strong opinion on what's best, but I have a feeling that if Weblate is opting to use Celery as it's scheduling mechanism, then it might be easier to just fulfill this requirement than maintaining our own sets of cron jobs in the longer term.

IMO we don't have the option to do all the background tasks by cronjobs, as we than would need to invest a lot of time to extract the information by Weblate, what is sent to Celery.

Anything against using Redis as a message broker to make celery work?

I think Redis is good solution, so let's go this way.

#5 Updated by drebs 23 days ago

  • Status changed from Confirmed to Resolved

#6 Updated by zen about 19 hours ago

  • Related to Bug #17340: Weblate: Check Celery settings for sane values added

Also available in: Atom PDF