Project

General

Profile

Bug #11092

Deal with November 2016 false positive scenarios

Added by bertagaz over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Continuous Integration
Target version:
Start date:
11/05/2016
Due date:
12/05/2016
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:

Description

As defined in #10993, check is there are still some of them and tag them as fragile if any (see #10288)


Related issues

Blocks Tails - Bug #11892: Sometimes the remote shell doesn't start because of missing initial Space when modifying the kernel cmdline In Progress 10/31/2016
Blocks Tails - Bug #11409: Deal with the 'Dogtail: warning: application may be hanging' bug Resolved 05/11/2016

History

#1 Updated by bertagaz over 3 years ago

  • Target version set to Tails_2.9.1

#2 Updated by intrigeri over 3 years ago

  • Subject changed from Deal with November false positive scenarios to Deal with November 2016 false positive scenarios

#3 Updated by intrigeri almost 3 years ago

  • Blocked by Bug #11409: Deal with the 'Dogtail: warning: application may be hanging' bug added

#4 Updated by intrigeri almost 3 years ago

  • Blocks Bug #11892: Sometimes the remote shell doesn't start because of missing initial Space when modifying the kernel cmdline added

#5 Updated by anonym almost 3 years ago

  • Target version changed from Tails_2.9.1 to Tails 2.10

#6 Updated by intrigeri over 2 years ago

  • Assignee changed from bertagaz to intrigeri

#7 Updated by intrigeri over 2 years ago

  • Blocked by deleted (Bug #11409: Deal with the 'Dogtail: warning: application may be hanging' bug)

#8 Updated by intrigeri over 2 years ago

  • Blocks Bug #11409: Deal with the 'Dogtail: warning: application may be hanging' bug added

#9 Updated by intrigeri over 2 years ago

  • % Done changed from 0 to 40

Initial triaging for stable done, 2 failures left to investigate further.

#10 Updated by intrigeri over 2 years ago

  • % Done changed from 40 to 80

Initial triaging for devel done, a few failures left to investigate further.

#11 Updated by intrigeri over 2 years ago

  • % Done changed from 80 to 90

Finished triaging and creating tickets. Left to do: mark tests as fragile & send a summary that will help re-prioritizing test suite robustness work.

#12 Updated by intrigeri over 2 years ago

  • Status changed from Confirmed to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 90 to 100

This is about October + November.

  • Parts of the test suite (and sometimes large parts of it) have been broken a few times during weeks, which made the analysis very long and painful, not to mention that it decreases the value (for developers) of running tests. IMO it's totally worth it to prioritize fixing temporary test suite breakage very high: we'll have to fix them anyway, and the more we wait, the more work we'll have to do to triage false positives, and the more value we lose from our test suite. Example problems: #11960, #11911, #11906, #11901, #11874
  • Recurring issues:
    • the usual ones: #11872, #11816, #11786; these are most likely bugs in Tails rather than in the test suite; some of them might be fixed for free in Stretch, we'll see. I guess the Foundations Team should add them to their list of high-prio bugs to fix or workaround in some way.
    • #11890 => marked as fragile
    • #12045, not sure if it's a bug in the test suite or in Tails Installer
    • #11508 => raised priority
  • Tickets I had to create: #12040, #12041, #12042, #12043, #12044, #12045, #12046 and #12047. That's a lot, but I doubt that they really are all new problems; likely we were lazy and didn't create them during the last shifts.

Also available in: Atom PDF