Project

General

Profile

Feature #15052

Adding pdf-redact-tools

Added by dachary about 1 year ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
12/12/2017
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
https://gitlab.com/dachary/tails/commits/feature/15052-pdf-redact-tools
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

In the [context of SecureDrop](https://github.com/freedomofpress/securedrop/issues/2643) there is a need for tools to handle PDF files, in addition to [MAT](https://0xacab.org/mat/mat).

I'm willing to add [pdf-redact-tools](https://github.com/firstlookmedia/pdf-redact-tools/graphs/contributors) to Debian GNU/Linux and to the backports so it can be included in tails eventually.

However, if adding pdf-redact-tools is something you do not want to ship by default for some reason, I will find another way to make it available for SecureDrop.

What do you think ?

0001-Add-pdf-redact-tools-to-tails-pre-installed-software.patch View (979 Bytes) dachary, 12/30/2017 07:30 PM

screenshot.png View (247 KB) dachary, 01/03/2018 08:34 PM


Related issues

Related to Tails - Bug #15398: Document pdf-redact-tools Rejected 03/12/2018
Blocked by Tails - Bug #15132: devel branch FTBFS since aufs-dkms 4.14 is in sid Resolved 12/29/2017

Associated revisions

Revision 520acf91 (diff)
Added by dachary about 1 year ago

Add pdf-redact-tools to tails pre-installed software

While there's definitely some overlap between pdf-redact-tools and
MAT's mission, the UX and functionality pdf-redact-tools provides
would not really fit in MAT.

It complements MAT functions and is useful to journalists working
with sensitive documents.

refs: #15052

Revision 4bfbe687 (diff)
Added by bertagaz about 1 year ago

Update homepage URL of pdf-redact-tools.

Refs: #15052

Revision aabc225d (diff)
Added by bertagaz about 1 year ago

Document usage of pdf-redact-tools to clean metadata from PDF.

Refs: #15052

Revision 79dc5702
Added by bertagaz about 1 year ago

Merge branch 'feature/15052-pdf-redact-tools' into devel

Fix-committed: #15052

History

#1 Updated by intrigeri about 1 year ago

  • Assignee set to dachary
  • QA Check set to Info Needed

In the [context of SecureDrop](https://github.com/freedomofpress/securedrop/issues/2643) there is a need for tools to handle PDF files, in addition to [MAT](https://0xacab.org/mat/mat).

I had a quick look. While there's definitely some overlap between pdf-redact-tools and MAT's mission, the UX and functionality pdf-redact-tools provides would not really fit in MAT, so it seems to be a valid candidate and not a duplicate tool.

I'm willing to add [pdf-redact-tools](https://github.com/firstlookmedia/pdf-redact-tools/graphs/contributors) to Debian GNU/Linux and to the backports so it can be included in tails eventually.

Much appreciated.

However, if adding pdf-redact-tools is something you do not want to ship by default for some reason, I will find another way to make it available for SecureDrop.

OK, let's see:

  • I'm assuming you wrote "by default" because you're talking about Tails devices that are used offline, in an air-gapped environment. Correct? FYI in the next major Tails release (3.6, 2018-03-13), our Additional software packages feature will work fully offline, and then the SecureDrop's air-gapped Tails device provisioning could prepare a persistent volume, pre-configured to install pdf-redact-tools, and with a persistent APT cache seeded to include pdf-redact-tools and its missing dependencies. Actually one of our major motivations to make this feature work offline was precisely to better support air-gapped usage scenarios. For this to work, pdf-redact-tools needs to be available in Debian (or in an additional APT repository configured in your persistent volume). Would this help you drop your "by default" requirement? If not, independently from pdf-redact-tools, I would personally love to better understand why this would not help in the general case :)
  • The PPA's dependencies are: Depends: python:any (<< 2.8), python, python:any (>= 2.7~), imagemagick, libimage-exiftool-perl. Are they correct? The easiest way to test this is in a minimal Debian chroot, e.g. with pbuilder or using debootstrap manually. If they are, cool: we have them all in Tails currently (until we manage to drop Python 2.x from the ISO, but we're not there yet). So adding pdf-redact-tools would basically have zero impact on the ISO size.

#2 Updated by dachary about 1 year ago

so it seems to be a valid candidate and not a duplicate tool.

Thanks for the quick assessment, this is encouraging :-)

#3 Updated by intrigeri about 1 year ago

Thanks for the quick assessment, this is encouraging :-)

:)

Meta: I've heard that SecureDrop is considering moving from Tails to QubesOS. I want to make sure this is done for good reasons (there are surely plenty of good reasons to use QubesOS!) and not primarily because our two teams failed at communicating wrt. your needs and how we should prioritize them. Let's not discuss this right here, but I'd love to read a summary on about the major issues SecureDrop developers and operators are experiencing with Tails.

#4 Updated by dachary about 1 year ago

I'm assuming you wrote "by default" because you're talking about Tails devices that are used offline, in an air-gapped environment. Correct?

I should have detailed a little more the SecureDrop use case, let me try to fix that. We are indeed talking about the installation and upgrade of an airgap machine based on tails. The journalists using SecureDrop and the SecureDrop admin helping them also use tails but it is connected to the net and therefore less challenging .

The journalists uses the airgap tails to work on documents they received and they need tools to do so, pdf-redact-tools is one of them but not the only one. There are dozens of SecureDrop installations, all using tails, and the vast majority use the tails with no additional software, even when tails is not airgap . Although they could install additional packages /#additional_software it turns out they do not. They are aware it is possible but maybe it is too complicated for them to feel comfortable ? Maybe they should be trained but the lack of resources prevents that from happening ? Whatever the reason the fact remains: it does not happen.

IMHO we (SecureDrop developers) need to make an effort upstream (i.e. tails + Debian GNU/Linux) to provide additional software without requiring SecureDrop admins to add them themselves to the vanilla tails key. This is why I filed this issue and this is what I mean by default. So the existing SecureDrop users will benefit from the new tool the next time they upgrade their tails key (airgap or not).

#5 Updated by intrigeri about 1 year ago

Although they could install additional packages /#additional_software it turns out they do not. They are aware it is possible but maybe it is too complicated for them to feel comfortable ? Maybe they should be trained but the lack of resources prevents that from happening ? Whatever the reason the fact remains: it does not happen.

I have good news for you: we have identified this usability problem a while ago (currently one needs to do stuff as root on the command line to use Additional Software Packages) and our plans for 2018 include fixing it. We have funding for this project so it'll definitely happen. See #14568 and subtasks + blueprints if you want to follow along :)

#6 Updated by dachary about 1 year ago

Thanks for the nice discussion :-)

#7 Updated by dachary about 1 year ago

Packaging of pdf-redact-tools is tracked upstream at https://github.com/firstlookmedia/pdf-redact-tools/issues/26

#8 Updated by dachary about 1 year ago

Now that pdf-redact-tools is in sid, I suppose the next step is to submit a patch to the devel branch, like so:

From de8dae347bbd49b241606cad9d83060ee33d246e Mon Sep 17 00:00:00 2001
From: Loic Dachary <loic@dachary.org>
Date: Sat, 30 Dec 2017 20:13:23 +0100
Subject: [PATCH] Add pdf-redact-tools to tails pre-installed software

While there's definitely some overlap between pdf-redact-tools and
MAT's mission, the UX and functionality pdf-redact-tools provides
would not really fit in MAT.

It complements MAT functions and is be useful to journalists working
with sensitive documents.

refs: #15052
---
 config/chroot_local-packageslists/tails-common.list | 1 +
 1 file changed, 1 insertion(+)

diff --git a/config/chroot_local-packageslists/tails-common.list b/config/chroot_local-packageslists/tails-common.list
index 98f8fe15cf..e6e654fad4 100644
--- a/config/chroot_local-packageslists/tails-common.list
+++ b/config/chroot_local-packageslists/tails-common.list
@@ -149,6 +149,7 @@ live-tools
 lvm2
 macchanger
 mat
+pdf-redact-tools
 mesa-utils
 monkeysign
 monkeysphere
-- 
2.11.0

Reading the contributor guide I found the merge policy from which I (hopefully ;-) got the branch name right. However it seems to be a guide for people who are trusted to push to the tails repository. Would you be so kind as to point me to the URL that explains the process for first time contributors ?

Cheers

#9 Updated by intrigeri about 1 year ago

Now that pdf-redact-tools is in sid, I suppose the next step is to submit a patch to the devel branch, like so:

Thanks! I'll take a first quick look (as part of my "Welcome new contributors" job) before I hand it over to the release manager for our next major release

Reading the contributor guide I found the merge policy from which I (hopefully ;-) got the branch name right. However it seems to be a guide for people who are trusted to push to the tails repository. Would you be so kind as to point me to the URL that explains the process for first time contributors ?

Sure: https://tails.boum.org/contribute/how/code/#submit

#10 Updated by intrigeri about 1 year ago

  • Status changed from New to In Progress
  • Assignee changed from dachary to intrigeri
  • Target version set to Tails_3.6
  • % Done changed from 0 to 20
  • QA Check changed from Info Needed to Ready for QA
  • Type of work changed from Debian to Code

#11 Updated by dachary about 1 year ago

Sure: https://tails.boum.org/contribute/how/code/#submit

Oh... I wonder how I missed it. Thanks for the kind reminder.

#12 Updated by intrigeri about 1 year ago

  • Blocked by Bug #15132: devel branch FTBFS since aufs-dkms 4.14 is in sid added

#13 Updated by intrigeri about 1 year ago

  • Assignee changed from intrigeri to dachary
  • QA Check changed from Ready for QA to Dev Needed
  • Feature Branch set to feature/15052-pdf-redact-tools

The branch FTBFS because our APT pinning prevents installing packages from sid unless an exception is explicitly added, see config/chroot_apt/preferences.

#14 Updated by dachary about 1 year ago

Updated the feature branch and included an exception to config/chroot_apt/preferences does that look ok ?

#15 Updated by intrigeri about 1 year ago

Updated the feature branch and included an exception to config/chroot_apt/preferences does that look ok ?

Yes, this does look better! Please follow the "submit your work" doc once you've successfully built an ISO and tested pdf-redact-tools in there. You'll probably have to merge the branch for #15132 in order to build an ISO because branches based on devel currently FTBFS.

Also, I think this deserves to be mentioned next to MAT in wiki/src/doc/about/features.mdwn.

#16 Updated by dachary about 1 year ago

built the image, ran it and verified it contains a working pdf-redact-tools

Also the feature branch was updated with a commit that includes a mention of pdf-redact-tools next to MAT.

What should I do next ?

#17 Updated by dachary about 1 year ago

For the record the pdf-redact-tools dependencies are imagemagick and libimage-exiftool-perl and python, all of which are already in tail. The pdf-readact-tools .deb is ~5kb.

#18 Updated by intrigeri about 1 year ago

built the image, ran it and verified it contains a working pdf-redact-tools

\o/

Also the feature branch was updated with a commit that includes a mention of pdf-redact-tools next to MAT.
What should I do next ?

Follow https://tails.boum.org/contribute/merge_policy/#submit :)

I think Target version should be the next major release (3.6).

#19 Updated by dachary about 1 year ago

  • Assignee deleted (dachary)
  • QA Check changed from Dev Needed to Ready for QA
  • Feature Branch changed from feature/15052-pdf-redact-tools to https://gitlab.com/dachary/tails/commits/feature/15052-pdf-redact-tools

#20 Updated by bertagaz about 1 year ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 20 to 100
  • QA Check changed from Ready for QA to Pass

Merged into devel, so it will be shiped in 3.6, congrats! I've taken the liberty to update the URL of the homepage of the project, which seems to have changed, and document a little bit more its usage.

#21 Updated by dachary about 1 year ago

Thanks a lot, much appreciated :-)

#22 Updated by dachary about 1 year ago

  • Status changed from Fix committed to In Progress

#23 Updated by bertagaz about 1 year ago

  • Status changed from In Progress to Fix committed

#24 Updated by sajolida 12 months ago

  • Related to Bug #15398: Document pdf-redact-tools added

#25 Updated by bertagaz 11 months ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF