Project

General

Profile

Bug #10496

Bug #10288: Fix newly identified issues to make our test suite more robust and faster

apt-get scenarios are fragile

Added by kytv about 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Test suite
Target version:
Start date:
11/06/2015
Due date:
% Done:

100%

Feature Branch:
test/10496-apt-get-scenarios-are-fragile
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

These tests could use some retrying with new tor circuits.

    [sudo] password for amnesia: W: GPG error: tor+http://ftp.us.debian.org wheezy-backports Release: The following signatures were invalid: BADSIG 8B48AD6246925553 Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>
    W: Failed to fetch tor+http://ftp.us.debian.org/debian/dists/testing/main/binary-i386/Packages  Hash Sum mismatch
    W: Failed to fetch tor+http://ftp.us.debian.org/debian/dists/testing/non-free/binary-i386/Packages  Hash Sum mismatch
    W: Failed to fetch tor+http://ftp.us.debian.org/debian/dists/testing/main/i18n/Translation-en  404  Not Found
    W: Failed to fetch tor+http://ftp.us.debian.org/debian/dists/wheezy-backports/contrib/binary-i386/Packages  404  Not Found
    E: Some index files failed to download. They have been ignored, or old ones used instead.

Of course, retrying doesn't won't guarantee that the test can succeed. The forthcoming freezable-apt repo OTOH combined with retrying might.

Associated revisions

Revision fc28e0be (diff)
Added by intrigeri about 4 years ago

Mark apt-get scenario as fragile.

Refs: #10496

Revision 3e3922cf (diff)
Added by bertagaz over 3 years ago

Un-fragilize apt-get scenario.

Let see how they behaves now in Jenkins.

Refs: #10496

Revision 3e06ca0c (diff)
Added by bertagaz over 3 years ago

Use retry_tor on apt-get update scenario.

Also lower a bit the timeout to 15 minutes.

Refs: #10496

Revision 9c562d31 (diff)
Added by bertagaz over 3 years ago

Use retry_tor on the apt install scenario.

Refs: #10496

Revision ec90cfc6 (diff)
Added by bertagaz over 3 years ago

Remove useless and racy condition.

Refs: #10496

Revision cf7be34f (diff)
Added by bertagaz over 3 years ago

We can run command as root in recovery_proc.

Refs: #10496

Revision 9c5122fa (diff)
Added by bertagaz over 3 years ago

Remove downloaded APT lists on retry_tor().

Refs: #10496

Revision 5f2124e6 (diff)
Added by bertagaz over 3 years ago

Make timeout clearer.

Refs: #10496

Revision a7eff831
Added by intrigeri over 3 years ago

Merge branch 'test/10496-apt-get-scenarios-are-fragile' into stable

Fix-committed: #10496

History

#1 Updated by anonym about 4 years ago

  • Assignee set to anonym

#2 Updated by anonym about 4 years ago

  • Target version set to 246

#3 Updated by anonym about 4 years ago

  • Assignee changed from anonym to kytv

#4 Updated by intrigeri about 4 years ago

The forthcoming freezable-apt repo OTOH combined with retrying might.

It will fix the root cause of the problem described here.

#5 Updated by sajolida about 4 years ago

  • Target version changed from 246 to Tails_2.0

#6 Updated by intrigeri about 4 years ago

  • Target version changed from Tails_2.0 to Tails_2.2

(We're going to mark as fragile all tests that depend on Tor to have bootstrapped for the moment => not so urgent. Waiting for freezable APT repo might make sense too.)

#7 Updated by intrigeri about 4 years ago

  • Type of work changed from Test to Code

#8 Updated by bertagaz almost 4 years ago

  • Assignee changed from kytv to bertagaz
  • Target version changed from Tails_2.2 to Tails_2.4

#9 Updated by bertagaz over 3 years ago

  • Target version changed from Tails_2.4 to Tails_2.5

#10 Updated by bertagaz over 3 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 20
  • Feature Branch set to test/10496-apt-get-scenarios-are-fragile

Pushed two commit that implements retry_tor in the APT scenario. Let just run it a moment in Jenkins.

#11 Updated by bertagaz over 3 years ago

  • Assignee changed from bertagaz to anonym
  • % Done changed from 20 to 60
  • QA Check set to Ready for QA

Works fine in Jenkins and $HOME. That's a quite tiny patch, shouldn't be hard and long to review, and it's ready for it IMHO.

#12 Updated by bertagaz over 3 years ago

Note that this ticket branch has been merged into bugfix/10494-retry-htpdate-with-more-fragile-tests, so it has also been tested in Jenkins in this other branch.

#13 Updated by intrigeri over 3 years ago

  • Assignee changed from anonym to bertagaz
  • QA Check changed from Ready for QA to Dev Needed

"Also lower a bit the timeout to 15 minutes" -> and indeed "15 minutes" is clearer than "900", so please revert to 15*60 in the code :)

Please drop the condition in step 'I kill the process "apt"' if $vm.has_process?("apt"), it's useless and racy.

In the recovery_proc you can just run commands as root, no need to make things more complicated using sudo here (as opposed to the piece of code that exercises what we expect real users to do).

Maybe the test would be more correct, i.e. closer to the documented retry_tor semantics, if the recovery_proc for apt update deleted the lists that may have been downloaded already?

#14 Updated by bertagaz over 3 years ago

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

"Also lower a bit the timeout to 15 minutes" -> and indeed "15 minutes" is clearer than "900", so please revert to 15*60 in the code :)

5f2124e

Please drop the condition in step 'I kill the process "apt"' if $vm.has_process?("apt"), it's useless and racy.

ec90cfc

In the recovery_proc you can just run commands as root, no need to make things more complicated using sudo here (as opposed to the piece of code that exercises what we expect real users to do).

cf7be34

Maybe the test would be more correct, i.e. closer to the documented retry_tor semantics, if the recovery_proc for apt update deleted the lists that may have been downloaded already?

Well, I wondered, but thought it was closer to the user experience if we didn't delete them. Anyway, that's 9c5122fa, feel free to do whatever you want of it depending on your opinion.

#15 Updated by intrigeri over 3 years ago

  • Assignee changed from intrigeri to anonym

In the recovery_proc you can just run commands as root, no need to make things more complicated using sudo here (as opposed to the piece of code that exercises what we expect real users to do).

cf7be34

execute_successfully, maybe? Consider it code-reviewed by me if you want to do that change before anonym's review.

Maybe the test would be more correct, i.e. closer to the documented retry_tor semantics, if the recovery_proc for apt update deleted the lists that may have been downloaded already?

Well, I wondered, but thought it was closer to the user experience if we didn't delete them. Anyway, that's 9c5122fa, feel free to do whatever you want of it depending on your opinion.

I'm not 100% sure. I'll let anonym make the call. If we keep it, we might want execute_successfully here, though.

Other than that, code review passes for me!

#16 Updated by intrigeri over 3 years ago

  • Assignee changed from anonym to intrigeri

#17 Updated by intrigeri over 3 years ago

  • Status changed from In Progress to 11
  • Assignee deleted (intrigeri)
  • % Done changed from 60 to 100
  • QA Check changed from Ready for QA to Pass

Merged, thanks!

#18 Updated by intrigeri over 3 years ago

  • Status changed from 11 to Resolved

Also available in: Atom PDF