Project

General

Profile

Bug #12169

Tails ISO has $stable-proposed-updates enabled when Debian/Tails release timing don't play nicely together

Added by intrigeri over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
01/24/2017
Due date:
% Done:

100%

Feature Branch:
bugfix/12169-disable-proposed-updates
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Additional Software Packages

Description

We see this in Tails 2.10. That's fine at ISO build time (we were using an APT snapshot that gave us essentially the latest Jessie point-release as a result) but it's not OK at runtime: it means that Tails users who install additional packages will get it before Debian users would.

If we don't do anything about it, this will be the case until next Tails major release (2.12). It's too late to fix that in 2.10, but IMO we should mitigate it in 2.11.

Maybe we need to make config/chroot_local-includes/lib/live/config/1500-reconfigure-APT disable proposed-updates sources?

Associated revisions

Revision f0e18dec (diff)
Added by anonym over 2 years ago

Disable -proposed-updates at boot time.

If a Debian point release happens right after a freeze but we have
decided to enable it before the freeze to get (at least most of) it,
then we get in the situation where -proposed-updates is enabled in the
final release, which we don't want. We only want it enabled at build
time.

Will-fix: #12169

Revision ff6e95bd (diff)
Added by anonym over 2 years ago

Test suite: test that we don't ship any -proposed-updates APT sources.

Refs: #12169

Revision ee16b49e
Added by intrigeri over 2 years ago

Merge remote-tracking branch 'origin/bugfix/12169-disable-proposed-updates' into stable (Fix-committed: #12169)

Revision 05a6629d (diff)
Added by intrigeri 8 months ago

Have the TR use the .packages file generated locally instead of one provided by the RM (refs: #12169)

Since I've removed the email from the RM to the TR, it may be that the TR
did not receive that file at all. And anyway, better use one generated
from source locally instead of one provided by the RM, which could allow
a compromised RM system to perform a bunch of attacks.

Revision efc439dc (diff)
Added by intrigeri 8 months ago

Clarify what we're verifying (refs: #12169)

Revision 65e89774 (diff)
Added by intrigeri 8 months ago

Simplify (refs: #12169)

We don't need to use the pad as an additional intermediary.

Revision c9241e59 (diff)
Added by intrigeri 8 months ago

Fix TOC (refs: #12169)

Revision 169eaa70 (diff)
Added by intrigeri 8 months ago

Clarify (refs: #12169)

To build is not sufficient to "reproduce".

Revision 0f685f15 (diff)
Added by intrigeri 8 months ago

Merge sections that are just steps of the same task (refs: #12169)

Revision 3f8dc28d (diff)
Added by intrigeri 8 months ago

Clarify heading (refs: #12169)

Revision aff22ac0 (diff)
Added by intrigeri 8 months ago

Disable verification of IUK reproducibility (refs: #12169)

The process would have to be automated a lot more for us to realistically do
this after every release. I'll make sure in Redmine that we come back to this
once we have improved this.

History

#1 Updated by intrigeri over 2 years ago

anonym, what do you think?

#2 Updated by anonym over 2 years ago

  • QA Check deleted (Info Needed)

intrigeri wrote:

We see this in Tails 2.10. That's fine at ISO build time (we were using an APT snapshot that gave us essentially the latest Jessie point-release as a result) but it's not OK at runtime: it means that Tails users who install additional packages will get it before Debian users would.

My bad, sorry! I should have filed a follow-up ticket once I merged the branch for #12124. I'll take care of the cleanup!

If we don't do anything about it, this will be the case until next Tails major release (2.12). It's too late to fix that in 2.10, but IMO we should mitigate it in 2.11.

Let's remove it ASAP!

Maybe we need to make config/chroot_local-includes/lib/live/config/1500-reconfigure-APT disable proposed-updates sources?

This sounds like an excellent suggestion! In fact, let's put as many sanity checks (with sane resolutions!) as possible there, so screw-ups like this can be tolerated.

#3 Updated by intrigeri over 2 years ago

I should have filed a follow-up ticket once I merged the branch for #12124.

The thing is, we couldn't simply revert "add -stable-updates" because 2.10 uses a snapshot of Debian that predates the Debian point release.
So I'm not quite sure how we could have dealt with this better, at least as long as one looks at the ISO build time topic only (which is what I did :)

Maybe we need to make config/chroot_local-includes/lib/live/config/1500-reconfigure-APT disable proposed-updates sources?

This sounds like an excellent suggestion! In fact, let's put as many sanity checks (with sane resolutions!) as possible there, so screw-ups like this can be tolerated.

Adding an automated test should be mostly trivial.

#4 Updated by anonym over 2 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • % Done changed from 0 to 40
  • QA Check set to Ready for QA
  • Feature Branch set to bugfix/12169-disable-proposed-updates

Ok, I think I got it! Please review'n'merge into stable and devel!

#5 Updated by intrigeri over 2 years ago

I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it's very unlikely that this is ever problematic, so I'll cross fingers and won't nitpick.

#6 Updated by intrigeri over 2 years ago

  • Assignee changed from intrigeri to anonym
  • % Done changed from 40 to 50

I figured it would take me less time to add an automated test than to test this branch by hand, so here's the deal: feel free to merge yourself after adding / running / debugging the following (untested!) automated test:

diff --git a/features/apt.feature b/features/apt.feature
index c835d27681..70c2413719 100644
--- a/features/apt.feature
+++ b/features/apt.feature
@@ -10,6 +10,7 @@ Feature: Installing packages through APT

   Scenario: APT sources are configured correctly
     Then the only hosts in APT sources are "vwakviie2ienjx6t.onion,sgvtcaew4bxjd7ln.onion,jenw7xbd6tf7vfhp.onion,sdscoq7snqtznauu.onion" 
+    And no proposed-updates APT suite is enabled

   @check_tor_leaks
   Scenario: Install packages using apt
diff --git a/features/step_definitions/apt.rb b/features/step_definitions/apt.rb
index 86f878d4a3..654d773914 100644
--- a/features/step_definitions/apt.rb
+++ b/features/step_definitions/apt.rb
@@ -11,6 +11,13 @@ Given /^the only hosts in APT sources are "([^"]*)"$/ do |hosts_str|
   }
 end

+Given /^no proposed-updates APT suite is enabled$/ do
+  $vm.file_content("/etc/apt/sources.list /etc/apt/sources.list.d/*").chomp.each_line { |line|
+    assert(not(line.match("-proposed-updates")),
+           "proposed-updates APT source found: #{line}")
+  }
+end
+
 When /^I configure APT to use non-onion sources$/ do
   script = <<-EOF
   use strict;

Fair enough? :)

#7 Updated by anonym over 2 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Ready for QA to Info Needed

intrigeri wrote:

I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it's very unlikely that this is ever problematic, so I'll cross fingers and won't nitpick.

I want to get it right, but I fail to see what is wrong. Please clarify!

I have pushed your test too, and I'd like to see it pass once on Jenkins before considering this branch for merging.

#8 Updated by intrigeri over 2 years ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Info Needed to Ready for QA

intrigeri wrote:

I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it's very unlikely that this is ever problematic, so I'll cross fingers and won't nitpick.

I want to get it right, but I fail to see what is wrong. Please clarify!

/-proposed-updates/d applies to any part of an APT source line, e.g. the hostname, and not only to what this sed regexp is meant to match (i.e. the suite component).

(BTW, some day I want to switch to deb822 style sources, that are a bit more structured and nicer to parse with higher-level tools. See sources.list(5) for details if curious :)

I have pushed your test too, and I'd like to see it pass once on Jenkins before considering this branch for merging.

Thanks!

#9 Updated by anonym over 2 years ago

  • Assignee changed from anonym to intrigeri

intrigeri wrote:

intrigeri wrote:

I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it's very unlikely that this is ever problematic, so I'll cross fingers and won't nitpick.

I want to get it right, but I fail to see what is wrong. Please clarify!

/-proposed-updates/d applies to any part of an APT source line, e.g. the hostname, and not only to what this sed regexp is meant to match (i.e. the suite component).

Ah, sure. Fixed now.

(BTW, some day I want to switch to deb822 style sources, that are a bit more structured and nicer to parse with higher-level tools. See sources.list(5) for details if curious :)

Looks cool, but I don't really think it will be easier to parse since you need to keep state across read lines => pretty much rules out grep, and requires sed that most people don't know about.

#10 Updated by intrigeri over 2 years ago

Looks cool, but I don't really think it will be easier to parse since you need to keep state across read lines => pretty much rules out grep, and requires sed that most people don't know about.

Hence "with higher-level tools" :)

#11 Updated by anonym over 2 years ago

intrigeri wrote:

Looks cool, but I don't really think it will be easier to parse since you need to keep state across read lines => pretty much rules out grep, and requires sed that most people don't know about.

Hence "with higher-level tools" :)

What, sed is not higher-level? :) Ok, then we'll need pretty powerful (arbitrary queries and manipulation) libraries with nice interfaces for at least shell, Ruby (test suite) and Python (shell replacement), but really for all languages where we interact with the APT lists.
</off-topic>

Also, I had to fixup your test since the semantics of VM.file_content() has changed in the base branch (stable) which eluded both of us since we didn't merge it in when running your test. :) Let's see what Jenkins thinks now.

#12 Updated by intrigeri over 2 years ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100

#13 Updated by intrigeri over 2 years ago

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#14 Updated by intrigeri over 2 years ago

  • Affected tool set to Additional Software Packages

#15 Updated by anonym over 2 years ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF