Bug #17343

[Weblate Database] switch from utf8 to utf8mb4 charset.

Added by hefee 4 months ago. Updated 5 days ago.

Target version:
Start date:
Due date:
% Done:


Feature Branch:
Type of work:
Affected tool:
Translation Platform


We are facing utf-8 encoing issues with the mariadb database.

those issues point to

and those pointing to the documentation:

the proposed solution is to switch to `utf8mb4`.

The database already using `utf8mb4` but not all tables. At least `trans_component` is using `utf8` instead of `utf8mb4`.

2019-12-12 15:11:40,251 - UWC - root(ERROR): Adding new component triggered by 'wiki/src/news/celebrating_10_years/' failed:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/mysql/", line 101, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib/python3/dist-packages/MySQLdb/", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python3/dist-packages/MySQLdb/", line 36, in defaulterrorhandler
    raise errorvalue
  File "/usr/lib/python3/dist-packages/MySQLdb/", line 217, in execute
    res = self._query(query)
  File "/usr/lib/python3/dist-packages/MySQLdb/", line 378, in _query
    rowcount = self._do_query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/", line 341, in _do_query
  File "/usr/lib/python3/dist-packages/MySQLdb/", line 280, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8D\\xA0 (...' for column 'source' at row 1")


#1 Updated by hefee 4 months ago

After we updated the database, we need to update the components:

2019-12-12 15:10:52,418 - UWC - root(INFO): Updated remote ffd5682881192702cd01e9986053b41a149eda82..54e2b58a7bb8276cb84de83e6b5eed938ee37cd1

#2 Updated by zen 4 months ago

Here's MariaDB's doc on Setting Character Sets and Collations.

#3 Updated by hefee 4 months ago

  • Assignee set to zen
  • Priority changed from Normal to Elevated
  • Target version set to Tails_4.2

#4 Updated by CyrilBrulebois 3 months ago

  • Target version changed from Tails_4.2 to Tails_4.3

#5 Updated by anonym about 2 months ago

  • Target version changed from Tails_4.3 to Tails_4.4

#6 Updated by zen about 1 month ago

If we do #16994 first, we might not need to do this one.

#7 Updated by CyrilBrulebois 20 days ago

  • Target version changed from Tails_4.4 to Tails_4.5

#8 Updated by zen 15 days ago

  • Due date set to 03/18/2020
  • Start date set to 03/18/2020

I plan to do this together with the change from MariaDB to PostgreSQL on 03/18/2020.

#9 Updated by zen 5 days ago

  • Due date changed from 03/18/2020 to 04/01/2020
  • Start date changed from 03/18/2020 to 04/01/2020

Pandemic context made me had to do something else on the day I had organized to do this, so I'm rescheduling for April 1st 2020.

Also available in: Atom PDF