Project

General

Profile

Feature #15291

Remove less popular packages that users could install themselves

Added by anonym over 1 year ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Category:
-
Target version:
Start date:
04/17/2018
Due date:
% Done:

100%

Feature Branch:
feature/15291-remove-software, https://salsa.debian.org/tails-team/tails/merge_requests/10
Type of work:
End-user documentation
Blueprint:
Starter:
Affected tool:

Description

... with Additional Software Packages version 2!


Related issues

Related to Tails - Feature #15281: Stack one single SquashFS diff when upgrading In Progress 03/07/2014
Related to Tails - Feature #15632: Consider distributing an alternative and minimal ISO image Rejected 06/03/2018 01/01/2019
Related to Tails - Bug #15536: Smaller ISO image Rejected 04/16/2018
Related to Tails - Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs Resolved 08/18/2018
Related to Tails - Bug #16290: Remove Scribus Resolved 01/05/2019
Related to Tails - Bug #16353: Decide what to do wrt. Traverso in Tails 4.0 Resolved 01/13/2019
Related to Tails - Bug #16387: hOpenpgp-tools doesn't work since GnuPG 2.1 Resolved 01/24/2019
Related to Tails - Bug #16571: Tor Browser does not list spell checker dictionaries installed via APT In Progress 03/18/2019
Related to Tails - Bug #15945: Drop Python 2 in Tails 5.0 In Progress 08/24/2015
Related to Tails - Feature #16399: Write release notes for 3.14 Resolved 01/29/2019
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed 03/22/2019
Blocks Tails - Feature #15941: Core work 2018Q4 → 2019Q2: Technical writing Resolved 09/11/2018

Associated revisions

Revision 2cee7949 (diff)
Added by CyrilBrulebois 3 months ago

Drop monkeysphere and msva-perl (refs: #15291).

Revision 9ffe2dce (diff)
Added by sajolida 3 months ago

Remove Gobby (refs: #15291)

Based on sajolida's work, updated by intrigeri to remove Gobby from the list of
Tails features.

Revision 13cc510f (diff)
Added by intrigeri 3 months ago

Test suite: remove tests that exercise Gobby or otherwise rely on it (refs: #15291)

We lose the "Explicitly torify-wrapped applications are using the default
SocksPort" and "Explicitly torsocks-wrapped applications are using the default
SocksPort" scenarios along the way. We could of course rewrite them to test
something else than Gobby but I see very little value in it: in practice,
they merely test that the default configuration of torsocks and torify
is what it has been since these pieces of software were created.

Revision 3ed99b06 (diff)
Added by intrigeri 3 months ago

Remove hopenpgp-tools, keyringer, libgfshare-bin, monkeysign, paperkey, pitivi, pdf-redact-tools, pwgen, traverso, and ssss (refs: #15291).

Revision febedc4f (diff)
Added by intrigeri 3 months ago

Remove leftovers of the Monkeysphere and msva-perl removal (refs: #15291)

Revision 1f0d3e25 (diff)
Added by intrigeri 3 months ago

Ensure UIDs and GIDs stability (refs: #15291)

Removing monkeysphere shifted many UIDs and GIDs, let's reset them
back to what they were in Tails 3.13.1.

Revision f173fab6
Added by CyrilBrulebois 2 months ago

Merge branch 'feature/15291-remove-software' into stable (Fix-committed: #15291).

History

#1 Updated by anonym over 1 year ago

  • Related to Feature #15281: Stack one single SquashFS diff when upgrading added

#2 Updated by anonym over 1 year ago

This is related to #15281 (1BigIUK) because of the larger IUKs. By removing software they won't ever be updated which otherwise would increase the IUK size.

#3 Updated by anonym over 1 year ago

  • Assignee set to sajolida
  • QA Check set to Info Needed

sajolida, would you like to work on a list of candidates for removal?

#4 Updated by anonym over 1 year ago

#5 Updated by anonym over 1 year ago

BTW, as far as #15281 is concerned, only packages with a history of frequent updates during Debian cycles (for security reasons or otherwise) matter; packages that are not upgraded through a Debian cycle does not impact the IUK in any way.

#6 Updated by sajolida over 1 year ago

  • Target version deleted (2018)
  • QA Check deleted (Info Needed)

We use "Target version: 2018" for stuff that we agreed to put on our roadmap. I don't think this was part of it.

In terms of timeline, I want to work on this only have we have a very solid implementation of Additional Software (#14568) and it has seen some adoption. A better bet would be 2019. By then we might have more research data that helps us select packages for removal.

But yes, I can do that as part of core UX.

#7 Updated by sajolida over 1 year ago

  • Feature Branch set to feature/15291-remove-software
  • Document edition
    • bookletimposer
    • pdf-redact-tools
    • poedit
  • Sound and video
    • traverso
    • pitivi
    • sound-juicer
    • brasero
    • cdrdao
  • OpenPGP tools
    • hopenpgp-tools
    • keyringer
    • paperkey
  • Misc
    • gobby
    • liferea
    • monkeysign
    • monkeysphere
    • msva-perl
    • nautilus-gtkhash
    • pwgen
    • ssss

#8 Updated by intrigeri about 1 year ago

Agreed with this list modulo I'd rather handle Liferea differently (#11082).

#9 Updated by sajolida about 1 year ago

  • Assignee deleted (sajolida)
  • Priority changed from Normal to Low

My branch now builds! The resulting ISO is 1121 MB while devel is 1168 MB. That's 4% less.

I'm a bit disappointed by the small difference :(

Note that devel has been broken for 12 days so maybe its size would be slightly different with a more recent build.

#10 Updated by intrigeri about 1 year ago

  • Related to Feature #15632: Consider distributing an alternative and minimal ISO image added

#11 Updated by Rover11 about 1 year ago

sajolida wrote:

  • Document edition
    • bookletimposer
    • pdf-redact-tools
    • poedit

...
_______________
Probably huge-size packages should be priorities for exclusion?
like: Inkscape, GIMP, Scribus, audacity

#12 Updated by intrigeri about 1 year ago

Probably huge-size packages should be priorities for exclusion?

I don't think that size should be the primary criterion but indeed, it does matter. IMO the criterion should be something like:

(size + frequency of security upgrades) / popularity

Size and popularity are obvious I guess. Frequency of security upgrades matter for two reasons:

  • packages that are often upgraded make automatic upgrades bigger for everyone;
  • whatever we ship in the ISO is essentially frozen, so users don't benefit of Debian security upgrades until our next release; while packages installed via ASP get upgraded on users' systems.

#13 Updated by sajolida about 1 year ago

#14 Updated by sajolida about 1 year ago

  • Assignee set to sajolida

I'll try prepare an ISO image with even less packages and see how much we gain in size.

#15 Updated by sajolida 12 months ago

I tried to remove even more packages with f4c21628f4 to see the impact on ISO size.

#16 Updated by sajolida 12 months ago

This one went down to 989315072 bytes vs 1217341440 in devel: 19% smaller than devel.

Now I'll experiment with an ISO with no internationalization :) Starting with b7b0122d57.

#17 Updated by sajolida 12 months ago

Removing fonts I get 929458176 instead of 1217341440 in devel: 23% less.

I'm now trying to remove precompiled locales in bda974fa5d.

#18 Updated by intrigeri 12 months ago

I'm glad you're doing all these experiments, it's great!

Let's keep in mind that some of these changes may increase boot/login time substantially, which kinda conflicts with our "Make it easier to switch between a Tails contextual identity and another identity outside of Tails" goal (that prompted me to start looking into low-hanging fruits wrt. boot time optimizations, stay tuned :)

#19 Updated by sajolida 12 months ago

Without locales-all the size is 921231360 vs 1217341440 in devel: 24% smaller than devel.

#20 Updated by u 11 months ago

  • Related to Feature #15543: Give a list of examples of nice software to install each time added

#21 Updated by u 11 months ago

#15543 was rejected but I think if we remove all these softwares that were previously shipped automatically, we absolutely need a list of recommended programs. I don't expect users to know what package they need to install to be able to sign keys (monkeysphere) or to burn a CD (brasero).

#22 Updated by sajolida 11 months ago

  • #15543 has been reopen.
  • I don't think we'll ever remove all these packages. I'm basically doing some experiment on the size of the ISO image, pushing it to the limit of what could be a super minimal (and limited) Tails image, which is borderline with #15632.

#23 Updated by intrigeri 11 months ago

I recommend you (temporarily) delete config/chroot_local-hooks/04-change-gids-and-uids from your branch so that it builds again. Since there's no plan to merge it, that should be fine :)

#24 Updated by intrigeri 11 months ago

  • Related to Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs added

#25 Updated by nodens 11 months ago

  • Target version set to 2019

#26 Updated by sajolida 10 months ago

Summary:

devel 1168 MB
remove less popular packages 02535f38b9 1121 MB -4%
remove document editing f4c21628f4 943 MB -19%
remove fonts b7b0122d57 886 MB -24%
remove locales bda974fa5d 878 MB -25%

I won't do any more experiment for now :)

#27 Updated by intrigeri 6 months ago

#28 Updated by intrigeri 6 months ago

  • Related to Bug #16353: Decide what to do wrt. Traverso in Tails 4.0 added

#29 Updated by goupille 6 months ago

  • Related to Bug #16387: hOpenpgp-tools doesn't work since GnuPG 2.1 added

#30 Updated by intrigeri 4 months ago

  • Related to Bug #16571: Tor Browser does not list spell checker dictionaries installed via APT added

#31 Updated by intrigeri 4 months ago

I think we can, and should:

  • ASAP, which could be in 4.0 or even earlier:
    • remove all the command line tools from your list: presumably, command line users are more at ease than others to find the package the need based on the name of the command
    • remove some of the other software on your list, when users who need them will know their name, e.g. Gobby, which makes it rather easy to find in Synaptic, as opposed to "I want to burn a CD, Synaptic gives me 150 results, which should I do?"
    • remove obsolete & unpopular software, such as pitivi, traverso and hopenpgp-tools
    • remove software that's not very usable in the context of Tails, especially if it's for a small part of our target userbase, such as monkeysign
    • remove highly specialized software that's mostly relevant for Tails contributors, such as keyringer and poedit
    • keep software that satisfies common use cases, when it's too hard to find what's the best option in Synaptic because there are too many results, e.g. Brasero and maybe others
    • keep LibreOffice, GIMP and Inkscape for now; Scribus is more specialized and is going away in 4.0 already
    • the above covers most of your initial list (#15291#note-7) already
  • Once we ship GNOME Software, assuming it will make it easier to find the best / most GNOMEish option, we can consider removing more graphical apps, e.g. some of f4c21628f4bdfa218fe7b6d9bbf17b4111507569.

#32 Updated by sajolida 4 months ago

  • Assignee changed from sajolida to intrigeri
  • QA Check set to Info Needed

I agree with your criteria. Criteria #2 and #6 are super smart!

Here is the list of all the packages I removed in my branch (removing localization and font packages):

audacity
bookletimposer
electrum
gimp
gnome-sound-recorder
gobby
hopenpgp-tools
inkscape
keyringer
liferea
pdf-redact-tools
monkeysign
monkeysphere
msva-perl
nautilus-gtkhash
paperkey
pitivi
poedit
pwgen
sane-utils
scribus
simple-scan
traverso
sound-juicer
ssss
brasero
cdrdao

I propose that we remove ASAP (3.14?):

gobby
hopenpgp-tools
keyringer
monkeysign
monkeysphere
msva-perl
paperkey
pitivi
pwgen
traverso
ssss
cdrdao
pdf-redact-tools

To address Ulrike's concern from #15543#note-21, I'll have to update some doc and maybe find how to recommend for packages for certain tasks.

I propose that we wait to see what happens with this 1st batch of removal before removing:

poedit
scribus

According to your criteria we should keep for now:

audacity
bookletimposer
electrum
gimp
gnome-sound-recorder
inkscape
liferea
nautilus-gtkhash
sane-utils
simple-scan
sound-juicer
brasero

I could probably have some fun working on this branch myself but otherwise it looks more like a mission for the Foundations Team, no?

#33 Updated by intrigeri 4 months ago

  • Related to Bug #15945: Drop Python 2 in Tails 5.0 added

#34 Updated by intrigeri 3 months ago

#35 Updated by intrigeri 3 months ago

#36 Updated by intrigeri 3 months ago

  • Target version changed from 2019 to Tails_3.14
  • QA Check changed from Info Needed to Dev Needed

I propose that we remove ASAP (3.14?):

I'll do that.

I propose that we wait to see what happens with this 1st batch of removal before removing:

Agreed.

I could probably have some fun working on this branch myself but otherwise it looks more like a mission for the Foundations Team, no?

Sure. I've renamed your branch to wip/feature/15291-remove-software-sajolida-experiment and will start a new feature/15291-remove-software branch that implements the above agreement. The wip/ prefix will make Jenkins ignore your branch for now; feel free to drop it whenever you want to resume experimenting with this :)

#37 Updated by intrigeri 3 months ago

I propose that we remove ASAP (3.14?):

I'll do that.

Err, except cdrdao: removing it would break some Brasero features and most users won't manage to find out they need to install cdrdao to get them back.

#38 Updated by intrigeri 3 months ago

  • Status changed from Confirmed to In Progress
  • QA Check deleted (Dev Needed)
  • Type of work changed from Research to Code

On top of the list agreed upon above, I'm also removing libgfshare-bin, for the same reason we're removing ssss.

My grep'ing while doing this work spotted some doc that needs updating:

  • doc/encryption_and_privacy/keyringer: remove this page? Or mention the fact one needs to install keyringer first?
  • doc/advanced_topics/paperkey: same as keyringer
  • doc/sensitive_documents/sound_and_video (pitivi, traverso)
  • doc/sensitive_documents/metadata
  • backlinks of the aforementioned pages, if any of them is removed

#39 Updated by intrigeri 3 months ago

#40 Updated by intrigeri 3 months ago

  • QA Check set to Ready for QA

#41 Updated by intrigeri 3 months ago

  • Assignee changed from intrigeri to sajolida
  • QA Check changed from Ready for QA to Dev Needed
  • Type of work changed from Code to End-user documentation

I've seen all our automated tests pass at least once; the code part is ready to be reviewed'n'merged by one of my fellow FT colleagues as soon as the doc is updated on this branch. So @sajolida, the floor is yours :) See my note above wrt. the bits of doc I've spotted that need updating.

#42 Updated by intrigeri 3 months ago

  • Blocks Feature #15941: Core work 2018Q4 → 2019Q2: Technical writing added

#43 Updated by intrigeri 3 months ago

  • Feature Branch changed from feature/15291-remove-software to feature/15291-remove-software, https://salsa.debian.org/tails-team/tails/merge_requests/10

#44 Updated by intrigeri 3 months ago

intrigeri wrote:

I've seen all our automated tests pass at least once; the code part is ready to be reviewed'n'merged by one of my fellow FT colleagues as soon as the doc is updated on this branch. So @sajolida, the floor is yours :) See my note above wrt. the bits of doc I've spotted that need updating.

My MR was approved on Salsa so feel free to merge straight into stable once you're happy with the doc update :)

#45 Updated by intrigeri 3 months ago

My MR was approved on Salsa […]

Glossary: MR = merge request (in the GitLab context; on GitHub it's PR = pull request).

#46 Updated by sajolida 3 months ago

Err, except cdrdao: removing it would break some Brasero features and most users won't manage to find out they need to install cdrdao to get them back.

I trust you more than me on these kinds of things but I listed cdrdao
because it's nowhere in the dependencies of Brasero:

Depends: libbrasero-media3-1 (= 3.12.1-4), libc6 (>= 2.4), libcairo2 (>=
1.2.4), libgdk-pixbuf2.0-0 (>= 2.22.0), libglib2.0-0 (>= 2.37.3),
libgstreamer-plugins-base1.0-0 (>= 1.0.0), libgstreamer1.0-0 (>= 1.0.0),
libgtk-3-0 (>= 3.0.0), libnautilus-extension1a (>= 3.21.92-3~),
libpango-1.0-0 (>= 1.14.0), libtotem-plparser18 (>= 3.10.0),
libtracker-sparql-1.0-0 (>= 0.10.0), libxml2 (>= 2.7.4),
gstreamer1.0-plugins-base (>= 0.11.92), gvfs, brasero-common (= 3.12.1-4)
Recommends: yelp, brasero-cdrkit
Suggests: vcdimager, libdvdcss2, tracker

#47 Updated by intrigeri 3 months ago

I trust you more than me on these kinds of things but I listed cdrdao because it's nowhere in the dependencies of Brasero:

Dependencies are transitive :) brasero depends on libbrasero-media3-1, which in turn has Recommends: cdrdao.

#48 Updated by sajolida 3 months ago

Dependencies are transitive :) brasero depends on libbrasero-media3-1, which in turn has Recommends: cdrdao.

Ha! I quickly checked level-2 dependencies of brasero but didn't think
that a "lib.*" package could depend on a non-"lib.*" package :)

#49 Updated by sajolida 2 months ago

  • Assignee changed from sajolida to cbrownstein

@cbrownstein: We're removing these packages:

gobby
hopenpgp-tools
keyringer
monkeysign
monkeysphere
msva-perl
paperkey
pitivi
pwgen
traverso
ssss
cdrdao
pdf-redact-tools

We should remove all mentions of them in the doc.

#50 Updated by cbrownstein 2 months ago

  • Assignee changed from cbrownstein to sajolida
  • QA Check changed from Dev Needed to Ready for QA

I've pushed a branch for review:

https://0xacab.org/cbrownstein/tails/tree/feature/15291-remove-software

It's based on origin/feature/15291-remove-software

Mentions were found by running git grep -i $package_name

#51 Updated by intrigeri 2 months ago

@cbrownstein or sajolida, it might be useful to check on #15291#note-38 that nothing I had listed escaped your git grep'ing :)

#52 Updated by sajolida 2 months ago

  • Assignee changed from sajolida to CyrilBrulebois
  • QA Check changed from Ready for QA to Pass

Cool!

I removed 1 extra thing with 198c9a3e13 (as per #15291#note-38).

@CyrilBrulebois: I didn't see a recent merge of this branch into stable, so I'll let you do whatever is needed for this to be in 3.14, right?

#53 Updated by CyrilBrulebois 2 months ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 0 to 100

#54 Updated by CyrilBrulebois 2 months ago

@sajolida:

I've just reviewed the changes between the merge base of this feature branch and stable, and just merged feature/15291-remove-software into stable.

#55 Updated by CyrilBrulebois about 2 months ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF