Project

General

Profile

Bug #17005

Upgrade to po4a 0.55 in Tails itself, in the Vagrant build box, on {translate,www}.lizard, and on RM's systems

Added by intrigeri 7 months ago. Updated 2 days ago.

Status:
In Progress
Priority:
Elevated
Assignee:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
tails.git:feature/17005-po4a-0.55, puppet-tails.git:feature/17005-po4a-0.55
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

po4a changed behavior between Stretch and Buster, e.g. wrt. linebreaks at the end of strings, and better extraction of bullet points. This is an improvement.

But this means that when updating with Buster's po4a a PO file that was generated with Stretch's po4a, tons of strings will be marked as fuzzy; and reciprocally. We've been bitten by this a couple times already, e.g. when a technical writer was using a recent version of po4a from Ubuntu, or when another used the one that's currently installed in Tails by our devel branch.

So we need to go through this upgrade in a coordinated way, taking the drawbacks into account.

Before we release 4.0, we need to upgrade to po4a 0.55 all systems that are responsible for generating or updating PO files:

  • Tails itself: some translators and technical writers work in there
  • Vagrant build box
  • www.lizard: ikiwiki refreshes PO files
  • translate.lizard: for the staging website
  • every RM's system: we update PO files as part of the release process

Tentative migration plan:

  1. Coordinate with stakeholders → emailed them on 2020-03-08; proposed to proceed with the deployment on March 14-15
  2. On a topic branch forked off master (feature/17005-po4a-0.55):
    1. Update our doc to require po4a 0.55 instead of the Stretch version in wiki/src/contribute/build/website.mdwn and wiki/src/contribute/release_process.mdwn.
    2. Upgrade po4a to the version from Buster in Tails itself (config/chroot_apt/preferences) and in our Vagrant box (vagrant/definitions/tails-builder/postinstall.sh).
    3. Refresh all website PO files with po4a 0.55. It'll make lots of strings fuzzy, which will have an immediate cost for translators, but on the long term it's worth it.
    4. Commit and push.
    5. To validate this branch and ease review, prepare and push a feature/17005-po4a-0.55-stable that will actually build on Jenkins, forked off feature/17005-po4a-0.55, with current stable merged into it.
    6. Review.
  3. On a topic branch for our Puppet code (feature/17005-po4a-0.55 in puppet-tails.git):
    1. Upgrade po4a on {translate,www}.lizard to the version from stretch-backports.
    2. Review
  4. Once everything is ready and reviewed:
    1. Soft-freeze our website
      1. Ask tech writers to avoid pushing until we're done here.
      2. Disable Weblate's automated push.
    2. Merge the Puppet branch, deploy, ensure po4a is upgraded.
    3. Merge the tails.git topic branch into master, refresh PO files again with po4a 0.55, commit.
    4. Merge master into stable, then stable into devel.
    5. Push master, stable, and devel.
    6. Re-enable Weblate automatic push.
    7. Ensure Weblate picks up the updates.
    8. Tell tech writers that they can push stuff again.
  5. Announce the change to people who maintain other systems themselves to build the website:
    1. people who translate in Git
    2. release managers
    3. technical writers
  6. Announce the change to Weblate translators

Ideally, to maximize the chances as many strings as possible get unfuzzied in the static version of our doc included in the first release that follows this change, we would merge all this early in a release cycle.


Related issues

Related to Tails - Bug #17127: Downgrade po4a to Stretch's 0.47-2 Resolved
Related to Tails - Bug #16868: Upgrade Vagrant box to Buster Resolved
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision 74286e27 (diff)
Added by intrigeri 27 days ago

Contributors doc: upgrade to po4a 0.55 (refs: #17005)

Revision 00897402 (diff)
Added by intrigeri 27 days ago

Tails images: upgrade to po4a 0.55 (refs: #17005)

Revision 17347ed0 (diff)
Added by intrigeri 27 days ago

Vagrant box: upgrade to po4a 0.55 (refs: #17005)

Revision 1c7a75b3 (diff)
Added by intrigeri 27 days ago

Website: refresh all PO files with po4a 0.55 (refs: #17005)

Revision a50071a4 (diff)
Added by intrigeri 26 days ago

Website: unfuzzy PO file that our test suite relies upon (refs: #17005)

"Scenario: The Report an Error launcher will open the support documentation in
supported non-English locales" relies on "Search the documentation" being
translated to "Die Dokumentation durchsuchen" in German.

History

#1 Updated by intrigeri 7 months ago

#2 Updated by intrigeri 7 months ago

I wrote "the earlier, the better", but let's not do this before 3.16 and 4.0~beta2 are out: if anything goes wrong, we don't want to disrupt the release process.

#3 Updated by intrigeri 7 months ago

  • Description updated (diff)

#4 Updated by intrigeri 7 months ago

If we don't manage to get this done in time for 4.0, we can downgrade po4a to the Stretch version in 4.0 and postpone.

#5 Updated by intrigeri 6 months ago

  • Related to Bug #17127: Downgrade po4a to Stretch's 0.47-2 added

#6 Updated by intrigeri 6 months ago

  • Target version changed from Tails_4.0 to Tails_4.1

intrigeri wrote:

If we don't manage to get this done in time for 4.0, we can downgrade po4a to the Stretch version in 4.0 and postpone.

We still have quite some non-trivial FT work to do in time for 4.0~rc1/4.0, that cannot be postponed, so to be on the safe side, I think we should postpone this one and apply the aforementioned workaround → #17127.

#7 Updated by intrigeri 6 months ago

  • Subject changed from Use the same version of po4a in Tails 4.0, on www.lizard, and on RM's systems to Upgrade to po4a 0.55 in Tails itself, on www.lizard, and on RM's systems

#8 Updated by intrigeri 6 months ago

  • Related to Bug #16868: Upgrade Vagrant box to Buster added

#9 Updated by intrigeri 6 months ago

  • Subject changed from Upgrade to po4a 0.55 in Tails itself, on www.lizard, and on RM's systems to Upgrade to po4a 0.55 in Tails itself, in the Vagrant build box, on www.lizard, and on RM's systems

(Once #16868 is merged, we'll have another place where we forcefully install po4a 0.47.)

#10 Updated by intrigeri 5 months ago

  • Target version deleted (Tails_4.1)

It would be nice if we could give translators the benefits of po4a 0.55 but have plenty of other important things to do.

#11 Updated by emmapeel 3 months ago

At the moment the instructions on the website ( https://tails.boum.org/contribute/build/website/ ) tell you to install po4a/stretch but when trying to do it from Tails 4.2.x, the server says something like: could not find po4a in stretch distribution.

Installing po4a works well, though.

#12 Updated by intrigeri 3 months ago

Hi @emmapeel,

At the moment the instructions on the website
( https://tails.boum.org/contribute/build/website/ ) tell you to
install po4a/stretch but when trying to do it from Tails 4.2.x, the
server says something like: could not find po4a in
stretch distribution.

AFAICT, the instructions in the "Build the website in Tails" section don't have this problem.
So I've updated that doc page a bit:

  • increase the chances Tails users follow the relevant instructions
  • improve a bit the non-Tails instructions (not relevant here, but it was cheap while I was at it)

Better?

#13 Updated by intrigeri 2 months ago

  • Subject changed from Upgrade to po4a 0.55 in Tails itself, in the Vagrant build box, on www.lizard, and on RM's systems to Upgrade to po4a 0.55 in Tails itself, in the Vagrant build box, on {translate,www}.lizard, and on RM's systems
  • Description updated (diff)

I think we should give up on developers unfuzzying the strings that this mass-change to PO files will make fuzzy:

  • This requirement is the reason why we've been delaying this upgrade so far and I suspect it would delay it for at least 5 more months.
  • The upgrade will make our translators' life easier because the new po4a handles newlines at the end of strings in a way that's more friendly (basically, it hides them from translators). Given the frequency at which we see inconsistent-trailing-newlines errors, i.e. at which translators get these newlines wrong, it seems clear that this will give us recurring benefits ⇒ IMO it's worth paying the migration cost up-front ASAP, even with the drawback that translators have to (trivially) unfuzzy a large number of newly-fuzzy strings.

I've updated the migration plan in the description accordingly.

#14 Updated by intrigeri 2 months ago

  • Description updated (diff)
  • Assignee set to intrigeri
  • Target version set to Tails_4.3

I'll try to get all the bits ready in time to merge them immediately after 4.3 is out.

#15 Updated by intrigeri about 2 months ago

  • Target version changed from Tails_4.3 to Tails_4.4

#16 Updated by CyrilBrulebois about 1 month ago

Hey, I was just wading through some tickets targetting 4.4, and I was wondering whether this might be merged in time for it, so that I can prepare my host environment accordingly?

#17 Updated by intrigeri about 1 month ago

Hey, I was just wading through some tickets targetting 4.4, and I was wondering whether this might be merged in time for it, so that I can prepare my host environment accordingly?

Thank you for caring. The plan is to merge this early in a release cycle, so that translators have as much time as we can give them to unfuzzy tons of strings.
So no, it won't get merged in 4.4. Best case, during this dev cycle, I'll prepare branches that can be merged soon after 4.4 is released.

#18 Updated by intrigeri 27 days ago

  • Description updated (diff)

#19 Updated by intrigeri 27 days ago

  • Status changed from Confirmed to In Progress

#20 Updated by intrigeri 27 days ago

  • Description updated (diff)
  • Feature Branch set to tails.git:feature/17005-po4a-0.55, puppet-tails.git:feature/17005-po4a-0.55

#21 Updated by CyrilBrulebois 23 days ago

  • Target version changed from Tails_4.4 to Tails_4.5

#22 Updated by intrigeri 21 days ago

  • Target version changed from Tails_4.5 to Tails_4.6

That's postponed to early-post-4.5 days, to avoid fiddling with the stable branch at a time when our RMs need to see it change as little as possible.

#23 Updated by hans 11 days ago

FYI we are currently in the process of porting f-droid.org to build entirely using Debian/buster packages, including po4a. There were some Markdown bugs fixed recently, post 0.55, that are very much worth having. I recommend using at least 0.58 from git. We have one last bug open, after that I'm going to push for a 0.58 release which can then go to buster-backports. This is the bug in question: https://github.com/mquinson/po4a/issues/196

These are the bugs that were fixed:

#24 Updated by intrigeri 10 days ago

Hi @hans,

There were some Markdown bugs fixed recently, post 0.55, that are very much worth having. I recommend using at least 0.58 from git. We have one last bug open, after that I'm going to push for a 0.58 release which can then go to buster-backports. This is the bug in question: https://github.com/mquinson/po4a/issues/196

These are the bugs that were fixed: […]

Great, thanks for the pointers!
I'm glad we're on the same boat here, even if we recently were not able to jump on other boats that would have been nice to share :)

These bug fixes indeed look potentially useful. On the Tails side, the improvements we care about most are in 0.55 so I think I'll first go through this upgrade (which is basically ready to go), and cross the 0.58+ bridge in another iteration, once 0.58 is in buster-backports and all relevant Tails persons run Buster or newer.

#25 Updated by hans 10 days ago

From what I've seen, adding the step to 0.55 in between will create new issues that you will have to migrate from, while jumping straight to 0.58 looks like the same amount of work as 0.55.

#26 Updated by intrigeri 9 days ago

Hi @hans,

From what I've seen, adding the step to 0.55 in between will create new issues that you will have to migrate from,

Thank you. This makes me very curious!
Are you referring to regressions in 0.55 compared to what we're currently using (0.47)?
Or anything else?

#27 Updated by hans 9 days ago

I wouldn't call them regressions, but https://github.com/mquinson/po4a/issues/194 is a good example. Basically, using 0.55 will give your translators lots of

```bash
and
```
strings to translate, while 0.58 will not. Going from 0.55 to 0.58 would then remove all those as strings.

#28 Updated by intrigeri 8 days ago

Hi @hans,

hans wrote:

I wouldn't call them regressions, but https://github.com/mquinson/po4a/issues/194 is a good example. Basically, using 0.55 will give your translators lots of

```bash
and
```
strings to translate, while 0.58 will not.

Thank you. Luckily (for the matter at hand), we don't use code fences.

I'll come back to it in a few weeks and we'll see what's the 0.58 status and whether we still need to support Stretch at that time.

#29 Updated by hans 3 days ago

FYI, there has been a bunch of other work on smoothing out the process of using the `po4a` command to do all the syncing. That new work means f-droid.org no longer needs a custom script to run `po4a`. The maintainer is soliciting feedback on improving the po4a.conf format, in case you are interested: https://lists.po4a.org/archives/list/devel@lists.po4a.org/thread/NUVSGUGKTYDTA66RSMHGE6ZZH3Q2CMNY/

#30 Updated by intrigeri 2 days ago

FYI, there has been a bunch of other work on smoothing out the process of using the `po4a` command to do all the syncing. That new work means f-droid.org no longer needs a custom script to run `po4a`. The maintainer is soliciting feedback on improving the po4a.conf format, […]

Thanks. Actually, we don't use the po4a(1p) program. We use the ikiwiki po plugin, which itself uses the Locale::Po4a::* Perl libraries.

Also available in: Atom PDF