Project

General

Profile

Bug #10497

Bug #10288: Fix newly identified issues to make our test suite more robust and faster

wait_until_tor_is_working helper is fragile

Added by anonym over 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Test suite
Target version:
Start date:
11/06/2015
Due date:
% Done:

100%

Feature Branch:
test/10497-tor-bootstrap-is-fragile
Type of work:
Research
Blueprint:
Starter:
Affected tool:

Description

We did what we could, basically, with #9516. The real fix is something like #10238.

f8e5fb9 and 1c48ca26 tagged these tests as fragile.


Related issues

Blocked by Tails - Feature #9521: Use the chutney Tor network simulator in our test suite Resolved 04/15/2016
Blocks Tails - Bug #10376: The "the Tor Browser loads the (startup page|Tails roadmap)" step is fragile Resolved 10/15/2015
Blocks Tails - Bug #10381: The "I open the address" steps are fragile Resolved 10/15/2015
Blocks Tails - Bug #10442: Totem "Watching a WebM video over HTTPS" test is fragile Confirmed 10/28/2015
Blocks Tails - Bug #10440: Time syncing scenarios are fragile Resolved 10/28/2015
Blocked by Tails - Bug #11401: robust_notification_wait sometimes opens the Applications menu which breaks tests Resolved 05/05/2016
Blocked by Tails - Bug #11398: Florence sometimes hides other windows, which breaks tests Resolved 05/05/2016

Associated revisions

Revision 1c48ca26 (diff)
Added by bertagaz over 3 years ago

Flag a online scenario in Jessie as fragile.

It depends on the `wait_until_tor_is_working` helper which is buggy.

Refs: #10497

Revision baa10dbf (diff)
Added by bertagaz over 3 years ago

Flag forgoten online scenarios as fragile.

Refs: #10497

Revision cd53d12c
Added by anonym over 3 years ago

Merge remote-tracking branch 'origin/bugfix/10487-improve-VM-and-OOM-settings-for-erasing-memory' into devel

Fix-committed: #9707, #10497

Revision 06634d46
Added by anonym about 3 years ago

Merge remote-tracking branches 'origin/test/10376-fix_startup-page_roadmap-test-is-fragile' and 'origin/test/11401-notification-wait-is-fragile' into testing

Fix-committed: #10376, #10497, #11398, #11401, #11464

History

#2 Updated by anonym over 3 years ago

  • Blocked by Bug #10238: Investigate why Tor sometimes is slow at or even fails bootstrapping added

#3 Updated by anonym over 3 years ago

  • Assignee changed from amette to anonym

Whoops, wrong assignee! :)

#4 Updated by anonym over 3 years ago

  • Parent task set to #10288

#5 Updated by anonym over 3 years ago

  • Target version set to Tails_2.3

#6 Updated by bertagaz over 3 years ago

  • Target version changed from Tails_2.3 to Tails_1.8

This bug has happened 19 times since Nov 6, 2015 and the last @fragile tagging, and has become one of the most important. It should probably be worked on quickly, so I'm updating its target version.

#7 Updated by intrigeri over 3 years ago

  • Type of work changed from Wait to Research

(Let's not encode the "blocked by" relationship in two different ways.)

#8 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_1.8 to Tails_2.0

(We're going to mark as fragile all tests that depend on Tor to have bootstrapped for the moment => not so urgent.)

#9 Updated by bertagaz over 3 years ago

We marked the related scenarios as fragile in f8e5fb9

#10 Updated by intrigeri over 3 years ago

  • Description updated (diff)

#11 Updated by anonym over 3 years ago

  • Status changed from Confirmed to Fix committed
  • % Done changed from 0 to 100

#12 Updated by intrigeri over 3 years ago

  • Status changed from Fix committed to Confirmed
  • % Done changed from 100 to 0

(Last change was caused by a typo.)

#13 Updated by anonym over 3 years ago

  • Target version changed from Tails_2.0 to Tails_2.2

#14 Updated by anonym over 3 years ago

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_2.2 to Tails_2.3

Hoping that #10238 will lead us to the solution. Otherwise simulating the Tor network locally with Chutney (#9521) is an alternative.

#15 Updated by intrigeri over 3 years ago

  • Priority changed from Elevated to Normal
  • Target version changed from Tails_2.3 to Tails_2.2
  • Feature Branch set to test/10497-tor-bootstrap-is-fragile

#16 Updated by intrigeri over 3 years ago

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_2.2 to Tails_2.3

#18 Updated by intrigeri over 3 years ago

  • Blocked by Feature #9521: Use the chutney Tor network simulator in our test suite added

#19 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_2.3 to Tails_2.4

#20 Updated by intrigeri about 3 years ago

  • Assignee changed from anonym to intrigeri

Will give it a try.

#21 Updated by intrigeri about 3 years ago

  • Blocked by deleted (Bug #10238: Investigate why Tor sometimes is slow at or even fails bootstrapping)

#22 Updated by intrigeri about 3 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to anonym
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA

(or should it be for bertagaz, since I took this over from anonym, and bertagaz committed to do reviews'n'merges for anonym's test suite stuff?)

So, I've made sure that all scenarios that have failed on this branch since we have chutney are flagged as fragile (on the devel branch), except the too general ones we didn't want to mark as fragile (e.g. those caused by #11398), then I've merged devel into this branch.

I'm confident that the test results will be good, but please give it a few days and look at https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10497-tor-bootstrap-is-fragile/ to make sure it's all right. If it isn't, please reassign to me.

#23 Updated by intrigeri about 3 years ago

  • Blocks Bug #10376: The "the Tor Browser loads the (startup page|Tails roadmap)" step is fragile added

#24 Updated by intrigeri about 3 years ago

  • Blocks Bug #10381: The "I open the address" steps are fragile added

#25 Updated by intrigeri about 3 years ago

  • Blocks Bug #10442: Totem "Watching a WebM video over HTTPS" test is fragile added

#26 Updated by intrigeri about 3 years ago

  • Blocks Bug #10440: Time syncing scenarios are fragile added

#27 Updated by intrigeri about 3 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Ready for QA to Dev Needed

Too many false positives on Jenkins, I'll flag them as @fragile and file tickets before re-submitting. Sorry for the noise!

#28 Updated by intrigeri about 3 years ago

  • Blocked by Bug #11401: robust_notification_wait sometimes opens the Applications menu which breaks tests added

#29 Updated by intrigeri about 3 years ago

  • Blocked by Bug #11398: Florence sometimes hides other windows, which breaks tests added

#30 Updated by intrigeri about 3 years ago

  • % Done changed from 50 to 60

Now that I've marked a few more tests as fragile, this seems to be pretty robust. However, this branch has the one for #11398 merged in, and it seems that this might introduce a new robustness problem, so I want to have that other one fixed before we merge test/10497-tor-bootstrap-is-fragile.

#31 Updated by intrigeri about 3 years ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Dev Needed to Ready for QA

#32 Updated by anonym about 3 years ago

  • Status changed from In Progress to Fix committed
  • Assignee changed from anonym to intrigeri
  • % Done changed from 60 to 100

It looks good => merged! However, I pushed the following tiny improvements on top which you might want to ack before closing this ticket properly:

22a81c3 Optimize scenario by not using the network.
3fc0368 Move feature-wide @fragile tag to the only affected scenario.
e036f60 Explain why the whole feature is @fragile.

#33 Updated by intrigeri about 3 years ago

  • QA Check changed from Ready for QA to Pass

22a81c3 Optimize scenario by not using the network.
3fc0368 Move feature-wide @fragile tag to the only affected scenario.
e036f60 Explain why the whole feature is @fragile.

ACK

#34 Updated by intrigeri about 3 years ago

  • Assignee deleted (intrigeri)

#35 Updated by anonym about 3 years ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF