Project

General

Profile

Bug #10441

Bug #10288: Fix newly identified issues to make our test suite more robust and faster

Synaptic test is fragile

Added by kytv about 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Test suite
Target version:
Start date:
10/28/2015
Due date:
% Done:

100%

Feature Branch:
test/10441-synaptic-test-is-fragile
Type of work:
Code
Blueprint:
Starter:
Affected tool:

00_23_42_Install_packages_using_Synaptic.png View (94.1 KB) kytv, 10/28/2015 07:19 AM


Related issues

Related to Tails - Bug #10412: Synaptic hangs after updating APT lists Resolved 10/23/2015
Blocks Tails - Feature #10900: "I should be able to install a package using Synaptic" step is fragile Resolved 01/12/2016
Copied to Tails - Bug #12586: Synaptic test is fragile on Stretch Rejected 10/28/2015

Associated revisions

Revision bbec8225 (diff)
Added by bertagaz over 3 years ago

Untag synaptic test as @fragile.

It has been marked so with a bunch of others in a single commit, so it
can't be reverted. We expect this scenario to be more robust in Jessie
and the synaptic segfault has gone. Let see how it behaves in Jenkins.

Refs: #10441

Revision c6e2de7e (diff)
Added by bertagaz over 3 years ago

Retry updating APT from Synaptic on network errors.

Do not rely on the Synaptic interface to test if the update finished,
given the lack of a clear notification, but test if APT processes are
running, restart if its too long.

Also brings better catch of Synaptic error window, as well as more robust
handling of Synaptic segfaults (if any) by restarting its process.

Refs: #10441

Revision c69e1935 (diff)
Added by bertagaz over 3 years ago

Use a better capture image for SynapticSearchWindow.

Refs: #10441

Revision 0c29fcbe (diff)
Added by bertagaz over 3 years ago

Retry installing a package with Synaptic on network error.

Refs: #10441

Revision 60faed9d (diff)
Added by bertagaz over 3 years ago

Wait for the packages database to be loaded by Synaptic.

That's the moment where Synaptic is done starting and unfreeze.

Refs: #10441

Revision feac0227 (diff)
Added by bertagaz over 3 years ago

Also check that synaptic is still running on APT update.

We've seen it segfaulting in the past, and this test wouldn't catch it
without this.

Refs: #10441

Revision 12bb9cb8 (diff)
Added by bertagaz over 3 years ago

Use test suite process killing facility.

Refs: #10441

Revision 743e5f20 (diff)
Added by bertagaz over 3 years ago

Simplify if condition.

Refs: #10441

Revision 885b8952 (diff)
Added by bertagaz over 3 years ago

Untag synaptic test as @fragile.

It has been marked so with a bunch of others in a single commit, so it
can't be reverted. We expect this scenario to be more robust in Jessie
and the synaptic segfault has gone. Let see how it behaves in Jenkins.

Refs: #10441

Revision d8355a90 (diff)
Added by bertagaz over 3 years ago

Retry updating APT from Synaptic on network errors.

Do not rely on the Synaptic interface to test if the update finished,
given the lack of a clear notification, but test if APT processes are
running, restart if its too long.

Also brings better catch of Synaptic error window, as well as more robust
handling of Synaptic segfaults (if any) by restarting its process.

Refs: #10441

Revision 6c84c87a (diff)
Added by bertagaz over 3 years ago

Use a better capture image for SynapticSearchWindow.

Refs: #10441

Revision f94bdc10 (diff)
Added by bertagaz over 3 years ago

Retry installing a package with Synaptic on network error.

Refs: #10441

Revision ffd74696 (diff)
Added by bertagaz over 3 years ago

Wait for the packages database to be loaded by Synaptic.

That's the moment where Synaptic is done starting and unfreeze.

Refs: #10441

Revision 96f0e8f8 (diff)
Added by bertagaz over 3 years ago

Also check that synaptic is still running on APT update.

We've seen it segfaulting in the past, and this test wouldn't catch it
without this.

Refs: #10441

Revision 4788e83b (diff)
Added by bertagaz over 3 years ago

Use test suite process killing facility.

Refs: #10441

Revision a2bbe2dc (diff)
Added by bertagaz over 3 years ago

Simplify if condition.

Refs: #10441

Revision d485b7b9 (diff)
Added by bertagaz over 3 years ago

Remove racy condition.

Refs: #10441

Revision a14bef56 (diff)
Added by bertagaz over 3 years ago

More explicit waiting times.

Refs: #10441

Revision 3572ed10 (diff)
Added by bertagaz over 3 years ago

Use test suite proccess checking helper.

Refs: #10441

Revision 7acad4ee (diff)
Added by bertagaz over 3 years ago

Use apt for consistency.

Refs: #10441

Revision 886be18c (diff)
Added by bertagaz over 3 years ago

Use vm.has_process everywhere, damnit!

Refs: #10441

Revision 2a88c643
Added by anonym over 3 years ago

Merge remote-tracking branch 'origin/test/10441-synaptic-test-is-fragile' into devel

Fix-committed: #10412, #10441

History

#1 Updated by kytv about 4 years ago

  • Status changed from New to Confirmed
  • Parent task set to #10288

#2 Updated by kytv about 4 years ago

  • Status changed from Confirmed to In Progress
  • Target version set to Tails_1.7
  • Feature Branch set to kytv/test/10441-synaptic-test-is-fragile

#3 Updated by kytv about 4 years ago

  • Target version changed from Tails_1.7 to Tails_1.8

#5 Updated by intrigeri about 4 years ago

(This ticket should not have a parent with a target version before its own one.)

#6 Updated by intrigeri about 4 years ago

  • Assignee set to kytv

#7 Updated by kytv about 4 years ago

  • Target version changed from Tails_1.8 to 246

#8 Updated by sajolida about 4 years ago

  • Target version changed from 246 to Tails_2.0

#9 Updated by intrigeri about 4 years ago

  • Target version changed from Tails_2.0 to Tails_2.2

(We're going to mark as fragile all tests that depend on Tor to have bootstrapped for the moment => not so urgent. Besides, the move to Jessie and freezable APT repo can change things a bit.)

#10 Updated by anonym almost 4 years ago

  • Blocks Feature #10900: "I should be able to install a package using Synaptic" step is fragile added

#11 Updated by anonym almost 4 years ago

  • Target version changed from Tails_2.2 to Tails_2.4

#12 Updated by anonym almost 4 years ago

  • Assignee changed from kytv to anonym

I'll try to improve this with Chutney (#9521) and/or dogtail (#7729) depending on exactly what the problem is.

#13 Updated by anonym over 3 years ago

  • Feature Branch changed from kytv/test/10441-synaptic-test-is-fragile to test/10441-synaptic-test-is-fragile

The attached screenshot shows that fetching some indexes from deb.tails.boum.org failed. That does not seem specific to synaptic. I've pushed the topic branch which removes the @fragile tag + merged devel. Let's see what jenkins thinks.

#14 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_2.4 to Tails_2.6

#15 Updated by bertagaz over 3 years ago

  • % Done changed from 0 to 40

anonym wrote:

The attached screenshot shows that fetching some indexes from deb.tails.boum.org failed. That does not seem specific to synaptic. I've pushed the topic branch which removes the @fragile tag + merged devel. Let's see what jenkins thinks.

It finds a few issue, as did the runs at home. I've pushed a few commits that should make this test more robust. Works well at home, let see what jenkins thinks about it (first Jenkins run is https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10441-synaptic-test-is-fragile/29/)

#16 Updated by bertagaz over 3 years ago

  • % Done changed from 40 to 50
  • QA Check set to Ready for QA

The commits I've pushed in the branch are working fine and recovered from network errors during my testing. Works fine in Jenkins too.

So I think it's ready for a review. #10412 should be reviewed at the same time, given there is a workaround for this one too in this branch.

#17 Updated by bertagaz over 3 years ago

Note that this ticket branch has been merged into bugfix/10494-retry-htpdate-with-more-fragile-tests, so it has also been tested in Jenkins in this other branch.

#18 Updated by intrigeri over 3 years ago

  • Related to Bug #10412: Synaptic hangs after updating APT lists added

#20 Updated by intrigeri over 3 years ago

  • Blocks Bug #10494: Retry htpdate when it fails added

#21 Updated by intrigeri over 3 years ago

  • Blocks deleted (Bug #10494: Retry htpdate when it fails)

#22 Updated by intrigeri over 3 years ago

  • Assignee changed from anonym to bertagaz
  • QA Check changed from Ready for QA to Dev Needed

Here's a quick initial code review, to ease anonym's job.

The Git history seems broken: e.g. I see 23658c0276af07403f5c2b11992a124d1875cbb2 that seems to be a rewritten version of 2bb720275e933f33f8f8d6900257d1d5ac34f122 that is is devel. If it's not too late, please fix this.

Here as well, please drop the if $vm.has_process?("synaptic") check when killing processes (if I'm not mistaken, it's racy and useless).

Please write 15*60 instead of 900, for clarity's sake.

!$vm.execute("pidof /usr/lib/apt/methods/tor+http").success? <-- don't we have a nice helper to do that?

Please use apt instead of apt-get, for better consistency.

#23 Updated by bertagaz over 3 years ago

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

The Git history seems broken: e.g. I see 23658c0276af07403f5c2b11992a124d1875cbb2 that seems to be a rewritten version of 2bb720275e933f33f8f8d6900257d1d5ac34f122 that is is devel. If it's not too late, please fix this.

Right, my bad. This branch was based on devel at first, so I had to rebase it on stable, might be why. Fixed anyway.

Here as well, please drop the if $vm.has_process?("synaptic") check when killing processes (if I'm not mistaken, it's racy and useless).

d485b7b

Please write 15*60 instead of 900, for clarity's sake.

a14bef5

!$vm.execute("pidof /usr/lib/apt/methods/tor+http").success? <-- don't we have a nice helper to do that?

Ooh, right. Discovered it after this commit I guess... Fixed in 3572ed1

Please use apt instead of apt-get, for better consistency.

7acad4e

#24 Updated by intrigeri over 3 years ago

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Ready for QA to Dev Needed

!$vm.execute("pidof /usr/lib/apt/methods/tor+http").success? <-- don't we have a nice helper to do that?

Ooh, right. Discovered it after this commit I guess... Fixed in 3572ed1

Cool. There's if !$vm.execute("pidof synaptic").success? left though.

#25 Updated by bertagaz over 3 years ago

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

Cool. There's if !$vm.execute("pidof synaptic").success? left though.

O_o ooops... fixed in 886be18, sorry for the noise.

And many thanks for the reviews too!

#26 Updated by intrigeri over 3 years ago

  • Assignee changed from intrigeri to anonym

#27 Updated by intrigeri over 3 years ago

  • Assignee changed from anonym to bertagaz
  • QA Check changed from Ready for QA to Dev Needed

Doesn't merge cleanly anymore => please merge current stable into the topic branch, resolve conflicts and resubmit for QA.

#28 Updated by bertagaz over 3 years ago

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

Doesn't merge cleanly anymore => please merge current stable into the topic branch, resolve conflicts and resubmit for QA.

Right, done!

#29 Updated by intrigeri over 3 years ago

  • Assignee changed from intrigeri to anonym

#30 Updated by anonym over 3 years ago

  • Status changed from In Progress to 11
  • Assignee deleted (anonym)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

Your stuff looks good! Go bert, go!

#31 Updated by anonym over 3 years ago

  • Status changed from 11 to Resolved

#32 Updated by intrigeri over 2 years ago

  • Copied to Bug #12586: Synaptic test is fragile on Stretch added

#33 Updated by intrigeri over 2 years ago

  • Copied to deleted (Bug #12586: Synaptic test is fragile on Stretch)

#34 Updated by intrigeri over 2 years ago

  • Related to Bug #12586: Synaptic test is fragile on Stretch added

#35 Updated by intrigeri over 2 years ago

  • Related to deleted (Bug #12586: Synaptic test is fragile on Stretch)

#36 Updated by intrigeri over 2 years ago

  • Copied to Bug #12586: Synaptic test is fragile on Stretch added

Also available in: Atom PDF