Project

General

Profile

Feature #9956

Consider replacing the additional fonts we ship with Noto

Added by alant almost 4 years ago. Updated 4 months ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
Internationalization
Target version:
-
Start date:
08/09/2015
Due date:
% Done:

10%

Feature Branch:
feature/15807-trimmed-and-consistent-l10n-support, greeter:feature/9956-noto
Type of work:
Research
Blueprint:
Starter:
Affected tool:

Description

Noto fonts (https://en.wikipedia.org/wiki/Noto_fonts) is a font that "designed to cover all the scripts encoded in the Unicode standard". According to Behdad Esfahbod in his talk "Fonts without borders" at GUADEC 2015 it's ready to use, and the default font in android.

We might want to consider replacing the additional fonts we ship by this one.

It is packaged in debian in fonts-noto, which is available in wheezy-backports, jessie ane unstable. Under unstable, I wasn't able to find a language in wikipedia that is not displaied with only Debian standard donts and that one installed.


Related issues

Related to Tails - Bug #13473: Missing common proprietary Arabic fonts make some PDF documents unreadable Rejected 07/16/2017
Related to Tails - Feature #16095: Curate the list of languages in Tails Greeter Confirmed 11/04/2018
Duplicates Tails - Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs Resolved 08/18/2018

Associated revisions

Revision 68d2156a (diff)
Added by intrigeri almost 4 years ago

Replace all locale-specific fonts with the Noto fonts collection.

Fetch them from sid: this is a quite recent and ambitious project (it aims at
covering all scripts encoded in Unicode!), and I suspect that the version
included in Jessie and wheezy-backports, that's 2 years, may be missing too
many glyphs.

According to https://www.google.com/design/spec/style/typography.html, "Since
the Ice Cream Sandwich release, Roboto has been the standard typeface on
Android. Since Froyo, Noto has been the standard typeface on Android for all
languages not covered by Roboto. Noto is also the standard typeface for all
languages on Chrome OS."

My understanding of that web page is that Noto should cover anything that our
other standard GNOME & Linux fonts don't, so we probably don't need to include
Roboto in addition.

Refs: #9956

Revision ed7a35df (diff)
Added by intrigeri 4 months ago

Replace all locale-specific fonts with the Noto fonts collection.

These fonts aims at covering all scripts encoded in Unicode. They're the
standard font on Android for all languages not covered by Roboto.

Install them from sid so we don't have to go through adjusting stuff for
Buster (most of binary packages were renamed and what kind of fonts each ships
has changed).

Refs: #9956

Revision 936f7289 (diff)
Added by intrigeri 4 months ago

Enable the feature-9956-noto APT overlay (refs: #9956).

History

#1 Updated by intrigeri almost 4 years ago

  • Assignee set to alant
  • QA Check set to Info Needed

Sounds like a great idea in principle. Now I'm slightly confused:

Can you please clarify?

#2 Updated by alant almost 4 years ago

  • Assignee deleted (alant)
  • QA Check deleted (Info Needed)

intrigeri wrote:

Can you please clarify?

I may have misunderstood the talk, which is here: http://videos.guadec.org/2015/Fonts%20without%20Borders/.

Anyway is written above, with only this custom fonts I wasn't able to find a letter that wouldn't display in wikipedia.

#3 Updated by intrigeri almost 4 years ago

  • Type of work changed from Discuss to Research

OK, so that's Research work: IMO we need clarifications before we can discuss this any further.
Whether it's the default on Android or not (and what GNOME's plans are) makes quite a difference.

#4 Updated by intrigeri almost 4 years ago

  • Subject changed from Consider replacing the additional fonts we whip by noto to Consider replacing the additional fonts we ship by Noto

#5 Updated by intrigeri almost 4 years ago

  • Subject changed from Consider replacing the additional fonts we ship by Noto to Consider replacing the additional fonts we ship with Noto
  • Category set to Internationalization
  • Status changed from New to In Progress
  • % Done changed from 0 to 10
  • Feature Branch set to feature/9956-noto

I was curious after watching that talk (see the commit message for details wrt. Noto vs. Roboto), so I gave it a quick try.

Size-wise, Noto would make the ISO noticeably bigger:

  • Tails 1.5.1: 942MB
  • feature/9956-noto: 973MB

... but of course we would be covering more scripts.

Next question: would this move introduce regressions for non-Latin scripts that we currently ship dedicated fonts for? (Rationale: supporting more scripts would be nice, even if nobody ever requested them to be supported, but let's avoid breaking existing supported use cases. And we can't rely on the Android / Chrome OS experience since we're shipping a different DE and probably different font rendering engines and/or settings.) IMO this can only be resolved by publishing a test ISO and issuing a call for testing.

#6 Updated by intrigeri almost 4 years ago

I've discussed this with the current Debian installer maintainer (thanks Cyril!).

The graphical d-i now uses fonts-droid (Download-Size: 3.5MB) as a way to cover scripts that are not supported by the other fonts it includes. It would be good to check if simply adding that font would substantially improve our script coverage, without making the ISO much bigger, and without having to go through the testing process I suggested earlier on this ticket.

To see what font is used for which language in the graphical d-i, debcheckout rootskel-gtk and then see src/usr/bin/gtk-set-font in there (current version). Note that DroidSansFallback.ttf (the only font that's shipped by fonts-android-udeb) isn't explicitly listed in there, it's simply used as a fallback for missing glyphs.

#8 Updated by u about 2 years ago

  • Assignee set to alant

So it looks to me that there is a possibility to test the branch which ships Noto, although I guess that this branch will need to be updated and merged with current master first.

From the discussion from debian-devel which intrigeri has pointed to, Debian also switches to Noto.

@alant, could you please clarify what you think should be done? Who can update and test the branch?

#9 Updated by intrigeri about 2 years ago

  • Related to Bug #13473: Missing common proprietary Arabic fonts make some PDF documents unreadable added

#10 Updated by u over 1 year ago

ping @alan?

#11 Updated by u 11 months ago

One more gently ping :)

#12 Updated by intrigeri 11 months ago

  • Related to Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs added

#13 Updated by intrigeri 11 months ago

kibi, if you could point us to the current list of fonts shipped with the graphical d-i, this would be sweet :)

#14 Updated by CyrilBrulebois 11 months ago

There you go, after a netboot-gtk build of the master branch in unstable:

(sid-amd64-devel)kibi@armor:~/debian-installer/installer$ egrep '\s+(ttf|fonts)-' build/dest/MANIFEST.udebs|sort -u
    fonts-android-udeb 1:6.0.1r16-1.1 all
    fonts-dejavu-mono-udeb 2.37-1 all
    fonts-dejavu-udeb 2.37-1 all
    fonts-farsiweb-udeb 0.4.dfsg-12 all
    fonts-freefont-udeb 20120503-8 all
    fonts-khmeros-udeb 5.0-7 all
    fonts-knda-udeb 1.3-3 all
    fonts-lao-udeb 0.0.20060226-9 all
    fonts-lohit-guru-udeb 2.91.2-1 all
    fonts-mlym-udeb 7.0-2 all
    fonts-noto-hinted-udeb 20171026-2 all
    fonts-sil-abyssinica-udeb 1.500-1 all
    fonts-sil-padauk-udeb 3.003-1 all
    fonts-taml-udeb 1.2.2-4 all
    fonts-telu-udeb 2.5.5-1 all
    fonts-thai-tlwg-udeb 1:0.6.5-1 all
    fonts-tibetan-machine-udeb 1.901b-5 all
    fonts-ukij-uyghur-udeb 20110217-3 all

(Oh, see? Every font we rely on is finally named fonts-*, woohoo.)

#15 Updated by CyrilBrulebois 11 months ago

Checking my debian-boot@ folder, a few pointers related to Noto: We switched from the ttf-cjk-compact-udeb hack to fonts-android-udeb at this point:
  • #794432: debian-installer: please switch from ttf-cjk-compact-udeb to fonts-android-udeb
Some fun, long fixed:
  • #853921: fonts-android-udeb: broken rendering for Korean glyphs in debian-installer

#16 Updated by intrigeri 11 months ago

There you go, after a netboot-gtk build of the master branch in unstable:

Great, thanks!

(Oh, see? Every font we rely on is finally named fonts-*, woohoo.)

Grml, it was so much better in the good old XFree86 days, without all this young cool kids nonsense :]

#17 Updated by intrigeri 11 months ago

Next step: try to replace our current list of fonts with those used by the graphical d-i, build an ISO, compare its size with the relevant baseline, boot & login in many potentially affected languages, spot issues, optionally take screenshots and attach them here. If this looks good enough we can issue a call for testing on tails-testers, Twitter, and perhaps even our blog :)

#18 Updated by intrigeri 11 months ago

  • Assignee deleted (alant)

Rationale: Alan has been pinged twice in the last year, his last activity here was 3 years ago, let's make it clear that this ticket needs a volunteer.

#19 Updated by intrigeri 4 months ago

  • Status changed from In Progress to Confirmed

#20 Updated by intrigeri 4 months ago

intrigeri wrote:

Quoting https://en.wikipedia.org/wiki/Roboto: "On Android, the Noto font is used for languages not supported by Roboto, including Chinese (simplified and traditional), Japanese, Korean, Thai and Hindi". So indeed, in the context of this ticket, where the matter at hand is non-Latin scripts, Roboto doesn't matter.

#21 Updated by intrigeri 4 months ago

  • Status changed from Confirmed to In Progress

#22 Updated by intrigeri 4 months ago

intrigeri wrote:

Next step: try to replace our current list of fonts with those used by the graphical d-i, build an ISO,

Nah, that's not gonna work, at least not in the context of this ticket: d-i uses Noto only for two languages, with udebs that have nothig to do with the debs we can install. So for now I'll refocus this ticket on replacing all non-Latin fonts we ship with Noto. We can come back to "copy whatever d-i does" if the Noto attempt fails.

But that part is relevant:

compare its size with the relevant baseline, boot & login in many potentially affected languages, spot issues, optionally take screenshots and attach them here. If this looks good enough we can issue a call for testing on tails-testers, Twitter, and perhaps even our blog :)

I've refreshed the topic branch and will do that.

#23 Updated by intrigeri 4 months ago

For context, I'm working on this now mostly in order to:

  • simplify the discussion on #15807: dropping fonts there has a broader impact than "removing support for Tails in language X": it also prevents displaying any document or web page in language X, regardless of what locale Tails is running in
  • simplify the discussion on #16095: hiding languages in the Greeter is fine but then, maybe users of these languages still should be able to read & write documents in their language without having to find out which font package they need to install, even if the Tails UI is in English
  • avoid having to maintain a long list of fonts accross Debian upgrades: there's quite some churn on the Debian front and we've done a very poor job at keeping our list up-to-date wrt. current best practices; I'd rather have one set of Noto fonts instead

#24 Updated by intrigeri 4 months ago

  • Assignee set to intrigeri
  • Target version set to Tails_3.14

compare its size with the relevant baseline,

I've refreshed the topic branch and will do that.

The resulting ISO is 1169M (vs. 1177M for Tails 3.12.1). GNOME sessions in zh_CN and ar_EG look OK, at first glance, to my unskilled eye.

Next steps:

boot & login in many potentially affected languages, spot issues, optionally take screenshots and attach them here. If this looks good enough we can issue a call for testing on tails-testers, Twitter, and perhaps even our blog :)

While we're at it, we could also remove:

  • fonts-lyx, but that removes pitivi, which is already on our list of candidates for removal on #15291. It's too small to bother here and we remove pitivi on #15291, this font package will magically go away.
  • ttf-unifont: it's installed only because it was added as a dependency of the Greeter in… 2011; I suspect it's not used anymore, or would not be used on this branch.

#25 Updated by intrigeri 4 months ago

  • Feature Branch changed from feature/9956-noto to feature/9956-noto, greeter:feature/9956-noto

#26 Updated by intrigeri 4 months ago

  • Related to Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs added

#27 Updated by intrigeri 4 months ago

  • Related to Feature #16095: Curate the list of languages in Tails Greeter added

#28 Updated by intrigeri 4 months ago

  • Related to deleted (Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs)

#29 Updated by intrigeri 4 months ago

  • Related to deleted (Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs)

#30 Updated by intrigeri 4 months ago

  • Blocks Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs added

#31 Updated by intrigeri 4 months ago

  • Feature Branch changed from feature/9956-noto, greeter:feature/9956-noto to feature/15807-trimmed-and-consistent-l10n-support, greeter:feature/9956-noto

#32 Updated by intrigeri 4 months ago

  • Blocks deleted (Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs)

#33 Updated by intrigeri 4 months ago

  • Status changed from In Progress to Duplicate
  • Assignee deleted (intrigeri)
  • Target version deleted (Tails_3.14)

To simplify I'm merging this with #15807.

#34 Updated by intrigeri 4 months ago

  • Duplicates Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs added

Also available in: Atom PDF