Project

General

Profile

Bug #14935

Future-proof the test suite vs future fundraising campaign headers

Added by intrigeri over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Test suite
Target version:
Start date:
11/08/2017
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
test/14935-dogtailify-tor-browser
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Browser

Description

"The persistent Tor Browser directory is usable" scenario is broken by the fundraising campaign header:

FindFailed: can not find TorBrowserSavedStartupPage.png in S(0)[0,0 1024x768]

I think this header depends on JS to work nicely, which we don't have on file:/// pages.

How about we look for the "Tor check" button instead?

02_26_23_The_persistent_Tor_Browser_directory_is_usable.png View (66.4 KB) intrigeri, 11/08/2017 03:07 PM

TorBrowserSaveOutputFileSelected.png View (705 Bytes) intrigeri, 02/21/2018 11:00 AM

00_14_26_The_persistent_Tor_Browser_directory_is_usable.png View (66 KB) intrigeri, 02/21/2018 11:03 AM


Related issues

Related to Tails - Bug #14923: devel branch FTBFS since torbrowser-launcher 0.2.8-4 was uploaded Resolved 11/06/2017
Blocks Tails - Feature #13241: Core work: Test suite maintenance Rejected 06/29/2017

Associated revisions

Revision 47df1d7f (diff)
Added by anonym over 1 year ago

Test suite: use Dogtail instead of image.

This image was invalidated during our Donation campaign was running,
and then by a Tor Browser bug (#14962). Bumping images is boring so
let's rely on Dogtail instead.

To make this scenario more robust we also switch to something else
than the start page, and restart the browser between save/open in
order to prevent Dogtail e.g. finding the old page that we just saved
instead of the saved version we just opened.

Refs: #14935, #14962

Revision 0241795e
Added by intrigeri about 1 year ago

Merge branch 'test/14935-dogtailify-tor-browser' into stable (Fix-committed: #14935)

History

#1 Updated by intrigeri over 1 year ago

  • Blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#2 Updated by intrigeri over 1 year ago

  • Related to Bug #14923: devel branch FTBFS since torbrowser-launcher 0.2.8-4 was uploaded added

#3 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_3.3 to Tails_3.5

#4 Updated by anonym over 1 year ago

Now it also is broken due to #14962.

#5 Updated by intrigeri over 1 year ago

  • Blocks deleted (Feature #13240: Core work 2017Q4: Test suite maintenance)

#6 Updated by intrigeri over 1 year ago

  • Blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#7 Updated by intrigeri over 1 year ago

#8 Updated by intrigeri over 1 year ago

  • Blocks deleted (Feature #13240: Core work 2017Q4: Test suite maintenance)

#9 Updated by u over 1 year ago

That sounds like a good plan, especially since we'll run other donation campaigns in the future :)

#10 Updated by anonym over 1 year ago

  • Target version changed from Tails_3.5 to Tails_3.6

#11 Updated by anonym over 1 year ago

  • Target version changed from Tails_3.6 to Tails_3.7

Since the banner was removed, this is now fixed. For now. So I'll just post-pone this so it won't happen again next time we do something similar.

How about we look for the "Tor check" button instead?

Since it's the Tor Browser we can use Dogtail, so I'll probably do that.

#12 Updated by anonym over 1 year ago

  • Subject changed from "The persistent Tor Browser directory is usable" test is broken by the fundraising campaign header to Future-proof the test suite vs future fundraising campaign headers
  • Description updated (diff)

#13 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_3.7 to Tails_3.6

Since the banner was removed, this is now fixed.

FYI Scenario: The persistent Tor Browser directory is usable fails for me here due to #14962. I understand it's not the same problem as the one this ticket was originally about, but nevertheless this scenario is still unusable to validate changes that affect the Tor Browser on the current devel branch and I think this should be fixed with a different timing than "before the next fundraising campaign" :)

#14 Updated by anonym over 1 year ago

intrigeri wrote:

Since the banner was removed, this is now fixed.

FYI Scenario: The persistent Tor Browser directory is usable fails for me here due to #14962. I understand it's not the same problem as the one this ticket was originally about, but nevertheless this scenario is still unusable to validate changes that affect the Tor Browser on the current devel branch and I think this should be fixed with a different timing than "before the next fundraising campaign" :)

It's not clear this is related. During the 3.6 cycle I'll try to verify, and postpone this to 3.7 or 3.8 if it is unrelated, otherwise work on this as high prio.

#15 Updated by anonym over 1 year ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA
  • Feature Branch set to test/14935-dogtailify-tor-browser

intrigeri wrote:
FYI Scenario: The persistent Tor Browser directory is usable fails for me here due to #14962. I understand it's not the same problem as the one this ticket was originally about, but nevertheless this scenario is still unusable to validate changes that affect the Tor Browser on the current devel branch and I think this should be fixed with a different timing than "before the next fundraising campaign" :)

Confirmed, and agreed. Fixed in the feature branch, please review'n'merge into stable! But the next step (And I can print the current page as "output.pdf"...) will fail due to #15336.

#16 Updated by intrigeri over 1 year ago

  • % Done changed from 50 to 60

Code review passes, testing.

#17 Updated by intrigeri over 1 year ago

This fails for me:

  Scenario: The persistent Tor Browser directory is usable                                                        # features/torified_browsing.feature:129
    Given I have started Tails without network from a USB drive with a persistent partition enabled and logged in # features/step_definitions/snapshots.rb:172
    And the network is plugged                                                                                    # features/step_definitions/common_steps.rb:79
    And Tor is ready                                                                                              # features/step_definitions/common_steps.rb:347
    And available upgrades have been checked                                                                      # features/step_definitions/common_steps.rb:379
    And all notifications have disappeared                                                                        # features/step_definitions/common_steps.rb:441
    Then the persistent Tor Browser directory exists                                                              # features/step_definitions/common_steps.rb:688
    And there is a GNOME bookmark for the persistent Tor Browser directory                                        # features/step_definitions/common_steps.rb:698
    When I start the Tor Browser                                                                                  # features/step_definitions/common_steps.rb:385
    And I open the address "https://tails.boum.org/about" in the Tor Browser                                      # features/step_definitions/browser.rb:66
    And I can save the current page as "index.html" to the persistent Tor Browser directory                       # features/step_definitions/common_steps.rb:742
      FindFailed: TorBrowserSaveOutputFileSelected.png: (50x34) seen at (203, 97) with 0.79 in S(0)[0,0 1024x768] E:Y, T:3.0
        Line 2757, in file Region.java
       (RuntimeError)
      features/torified_browsing.feature:139:in `And I can save the current page as "index.html" to the persistent Tor Browser directory'

I'm attaching the sikuli candidate.

I think that "the Tor Browser loads the startup page" waited long enough for the page title to be loaded, while "I open the address" does not necessarily (#11592), which can be noticed with a slow enough Internet connection such as mine. This looks like a race condition since on the failure screenshot we can see the proper title in the window titlebar. Maybe workaround this (that is, probably <=> workaround #11592) by adding "Tails - About" has loaded in the Tor Browser?

#18 Updated by anonym about 1 year ago

  • Priority changed from Normal to Elevated

I'll try to finish this one before bert imports the next Tor Browser (i.e. around March 9).

#19 Updated by bertagaz about 1 year ago

  • Target version changed from Tails_3.6 to Tails_3.7

#20 Updated by anonym about 1 year ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

I merged in stable and resolved the conflicts, which fixes you issue in #14935#note-17.

#21 Updated by intrigeri about 1 year ago

anonym wrote:

I merged in stable and resolved the conflicts, which fixes you issue in #14935#note-17.

I don't get how this fixed my problem but whatever: OK, will test this!

#22 Updated by intrigeri about 1 year ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 60 to 100

#23 Updated by intrigeri about 1 year ago

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#24 Updated by bertagaz about 1 year ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF