Project

General

Profile

Bug #17343

[Weblate Database] switch from utf8 to utf8mb4 charset.

Added by hefee about 1 month ago. Updated 11 days ago.

Status:
Confirmed
Priority:
Elevated
Assignee:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:
Translation Platform

Description

We are facing utf-8 encoing issues with the mariadb database.

those issues point to
https://github.com/WeblateOrg/weblate/issues/2218
https://github.com/WeblateOrg/weblate/issues/1054

and those pointing to the documentation:

https://docs.weblate.org/en/weblate-3.5.1/admin/install.html#mysql-or-mariadb

the proposed solution is to switch to `utf8mb4`.

The database already using `utf8mb4` but not all tables. At least `trans_component` is using `utf8` instead of `utf8mb4`.

2019-12-12 15:11:40,251 - UWC - root(ERROR): Adding new component triggered by 'wiki/src/news/celebrating_10_years/mafe.de.po' failed:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/mysql/base.py", line 101, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8D\\xA0 (...' for column 'source' at row 1")

History

#1 Updated by hefee about 1 month ago

After we updated the database, we need to update the components:

2019-12-12 15:10:52,418 - UWC - root(INFO): Updated remote ffd5682881192702cd01e9986053b41a149eda82..54e2b58a7bb8276cb84de83e6b5eed938ee37cd1

#2 Updated by zen about 1 month ago

Here's MariaDB's doc on Setting Character Sets and Collations.

#3 Updated by hefee about 1 month ago

  • Assignee set to zen
  • Priority changed from Normal to Elevated
  • Target version set to Tails_4.2

#4 Updated by CyrilBrulebois 11 days ago

  • Target version changed from Tails_4.2 to Tails_4.3

Also available in: Atom PDF