Project

General

Profile

Bug #14929

'I can view a file stored in "~/Tor Browser" but not in ~/.gnupg' scenario is broken

Added by intrigeri almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Test suite
Target version:
Start date:
11/07/2017
Due date:
% Done:

100%

Feature Branch:
test/improvements-from-buster-sprint
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Browser

Description

This makes it more painful than needed to evaluate my work on #14923 because this scenario is precisely the one I need.

On current devel branch:

14:37:18   Scenario: I can view a file stored in "~/Tor Browser" but not in ~/.gnupg                                                               # features/torified_browsing.feature:64
14:38:32     Given I have started Tails from DVD and logged in and the network is connected                                                        # features/step_definitions/snapshots.rb:172
14:38:32     And I copy "/usr/share/synaptic/html/index.html" to "/home/amnesia/Tor Browser/synaptic.html" as user "amnesia"                       # features/step_definitions/common_steps.rb:637
14:38:32     And I copy "/usr/share/synaptic/html/index.html" to "/home/amnesia/.gnupg/synaptic.html" as user "amnesia"                            # features/step_definitions/common_steps.rb:637
14:38:32     And I copy "/usr/share/synaptic/html/index.html" to "/tmp/synaptic.html" as user "amnesia"                                            # features/step_definitions/common_steps.rb:637
14:38:32     Then the file "/home/amnesia/.gnupg/synaptic.html" exists                                                                             # features/step_definitions/common_steps.rb:615
14:38:32     And the file "/lib/live/mount/overlay/home/amnesia/.gnupg/synaptic.html" exists                                                       # features/step_definitions/common_steps.rb:615
14:38:32     And the file "/live/overlay/home/amnesia/.gnupg/synaptic.html" exists                                                                 # features/step_definitions/common_steps.rb:615
14:38:32     And the file "/tmp/synaptic.html" exists                                                                                              # features/step_definitions/common_steps.rb:615
14:38:32     Given I start monitoring the AppArmor log of "/usr/local/lib/tor-browser/firefox"                                                     # features/step_definitions/common_steps.rb:858
14:38:32     When I start the Tor Browser                                                                                                          # features/step_definitions/common_steps.rb:385
14:38:32     And the Tor Browser loads the startup page                                                                                            # features/step_definitions/common_steps.rb:405
14:38:32     And I open the address "file:///home/amnesia/Tor Browser/synaptic.html" in the Tor Browser                                            # features/step_definitions/browser.rb:70
14:38:32     Then I see "TorBrowserSynapticManual.png" after at most 5 seconds                                                                     # features/step_definitions/common_steps.rb:456
14:38:32     And AppArmor has not denied "/usr/local/lib/tor-browser/firefox" from opening "/home/amnesia/Tor Browser/synaptic.html"               # features/step_definitions/common_steps.rb:870
14:38:32     Given I restart monitoring the AppArmor log of "/usr/local/lib/tor-browser/firefox"                                                   # features/step_definitions/common_steps.rb:858
14:38:32     When I open the address "file:///home/amnesia/.gnupg/synaptic.html" in the Tor Browser                                                # features/step_definitions/browser.rb:70
14:38:32     Then I do not see "TorBrowserSynapticManual.png" after at most 5 seconds                                                              # features/step_definitions/common_steps.rb:456
14:38:32       FindFailed: can not find TorBrowserSynapticManual.png in S(0)[0,0 1024x768]
14:38:32         Line 2171, in file Region.java
14:38:32        (RuntimeError)
14:38:32       ./features/support/helpers/sikuli_helper.rb:95:in `findfailed_hook'
14:38:32       ./features/support/helpers/sikuli_helper.rb:146:in `rescue in block (2 levels) in new'
14:38:32       ./features/support/helpers/sikuli_helper.rb:139:in `/^I (do not )?see "([^"]*)" after at most (\d+) seconds$/'
14:38:32       features/torified_browsing.feature:81:in `Then I do not see "TorBrowserSynapticManual.png" after at most 5 seconds'

At first glance it looks like the recent changes to FindFailed might have broken the case when we're trying not to find a picture on screen.


Related issues

Related to Tails - Bug #14874: "Failing to add a certificate to Pidgin" test scenario always fails on devel since Sikuli fuzzy matching was implemented Resolved 10/22/2017
Blocks Tails - Feature #13240: Core work 2017Q4: Test suite maintenance Resolved 06/29/2017
Blocks Tails - Feature #14817: Test suite improvements from buster sprint 1 Resolved 10/22/2017

Associated revisions

Revision 94d51da6 (diff)
Added by anonym almost 2 years ago

Test suite: fix the "I do not see ..." step's case.

The recently introduced always-on Sikuli image candidate search makes
wait() not work so great when expected to fail (as in the "do not"
case of this step) so let's use waitVanish() instead.

And, while we're at it, let's not use the FindFailed override for
waitVanish() -- it was purely a mistake that it was overridden!

Will-fix: #14929

Revision f018bc11 (diff)
Added by anonym almost 2 years ago

Test suite: fix the "I do not see ..." step's case.

The recently introduced always-on Sikuli image candidate search makes
wait() not work so great when expected to fail (as in the "do not"
case of this step) so let's use waitVanish() instead.

And, while we're at it, let's not use the FindFailed override for
waitVanish() -- it was purely a mistake that it was overridden!

Will-fix: #14929

History

#1 Updated by intrigeri almost 2 years ago

  • Blocks Feature #14814: Analyze test suite results on Buster: 2017-10 sprint added

#2 Updated by intrigeri almost 2 years ago

  • Blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#3 Updated by intrigeri almost 2 years ago

  • Related to Bug #14874: "Failing to add a certificate to Pidgin" test scenario always fails on devel since Sikuli fuzzy matching was implemented added

#4 Updated by intrigeri almost 2 years ago

  • Target version changed from Tails_3.3 to Tails_3.5

#5 Updated by intrigeri almost 2 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check set to Ready for QA
  • Feature Branch set to test/improvements-from-buster-sprint

On test/improvements-from-buster-sprint, where this code is being worked on, this fails too but the symptoms are different:

    When I open the address "file:///home/amnesia/.gnupg/synaptic.html" in the Tor Browser                                                # features/step_definitions/browser.rb:70
03:10:44.198112168: Found fuzzy candidate picture for TorBrowserSynapticManual.png with similarity 0.4
    Then I do not see "TorBrowserSynapticManual.png" after at most 5 seconds                                                              # features/step_definitions/common_steps.rb:456
      found 'TorBrowserSynapticManual.png' while expecting not to (RuntimeError)
      features/torified_browsing.feature:81:in `Then I do not see "TorBrowserSynapticManual.png" after at most 5 seconds'

So this does look like a regression caused by fuzzy-matching, that's not fixed in the topic branch.

#6 Updated by intrigeri almost 2 years ago

  • Blocks Feature #14817: Test suite improvements from buster sprint 1 added

#7 Updated by intrigeri almost 2 years ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed
  • Feature Branch deleted (test/improvements-from-buster-sprint)

Feel free to fix this directly on test/improvements-from-buster-sprint if you prefer a greater granularity :)

#8 Updated by anonym almost 2 years ago

intrigeri wrote:

Feel free to fix this directly on test/improvements-from-buster-sprint if you prefer a greater granularity :)

I fixed it in devel already: 94d51da6b8df96f8f4d81cc022fa9289c5f78c07

#9 Updated by anonym almost 2 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

#10 Updated by intrigeri almost 2 years ago

  • % Done changed from 0 to 60

LGTM, will now test.

#11 Updated by intrigeri almost 2 years ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Info Needed

anonym wrote:

I fixed it in devel already: 94d51da6b8df96f8f4d81cc022fa9289c5f78c07

I still see this regression on test/improvements-from-buster-sprint because it's based on stable. Any reason not to fix it in stable as well? This would help me increase confidence in my tests of test/improvements-from-buster-sprint, not mentioning fixing regressions in stable helps building confidence in the fact we're always ready to ship an emergency release :)

#12 Updated by anonym almost 2 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Info Needed to Ready for QA
  • Feature Branch set to test/improvements-from-buster-sprint

intrigeri wrote:

anonym wrote:

I fixed it in devel already: 94d51da6b8df96f8f4d81cc022fa9289c5f78c07

I still see this regression on test/improvements-from-buster-sprint because it's based on stable. Any reason not to fix it in stable as well? This would help me increase confidence in my tests of test/improvements-from-buster-sprint, not mentioning fixing regressions in stable helps building confidence in the fact we're always ready to ship an emergency release :)

I was confused. I cherry-picked the fix into test/improvements-from-buster-sprint and merged stable, so the next results should be good!

#13 Updated by intrigeri almost 2 years ago

I've started another full test suite run (this is @fragile so Jenkins does not help) with a branch based on current stable to double-check.

#14 Updated by intrigeri almost 2 years ago

  • Status changed from In Progress to Fix committed
  • Assignee deleted (intrigeri)
  • % Done changed from 60 to 100
  • QA Check changed from Ready for QA to Pass

#15 Updated by intrigeri over 1 year ago

  • Blocks deleted (Feature #14814: Analyze test suite results on Buster: 2017-10 sprint)

#16 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_3.5 to Tails_3.4

#17 Updated by anonym over 1 year ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF