Project

General

Profile

Bug #10994

Bug #10288: Fix newly identified issues to make our test suite more robust and faster

"I can view and print a PDF file" scenarios are fragile

Added by intrigeri over 3 years ago. Updated 6 months ago.

Status:
Confirmed
Priority:
Normal
Assignee:
-
Category:
Test suite
Target version:
-
Start date:
01/25/2016
Due date:
% Done:

30%

Feature Branch:
wip/test/10994-fragile-view-and-print-PDF
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

I've seen "Scenario: I can view and print a PDF file stored in /usr/share" and "Scenario: I can view and print a PDF file stored in non-persistent /home/amnesia" fail on Jenkins with a screenshot that looks like it has succeeded, see e.g. the attached file (that comes from https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/117/).

Debug log:

  Scenario: I can view and print a PDF file stored in non-persistent /home/amnesia              # features/evince.feature:14
calling as root: echo 'hello?'
call returned: [0, "hello?\n", ""]
[log] CLICK on (1024,384)
calling as root: /sbin/ifconfig eth0 | grep -q 'inet addr'
call returned: [1, "", ""]
calling as root: date -s '@1451226453'
call returned: [0, "Sun Dec 27 14:27:33 UTC 2015\n", ""]
    Given I have started Tails from DVD without network and logged in                           # features/step_definitions/snapshots.rb:201
calling as amnesia: cp "/usr/share/cups/data/default-testpage.pdf" "/home/amnesia" 
call returned: [0, "", ""]
    And I copy "/usr/share/cups/data/default-testpage.pdf" to "/home/amnesia" as user "amnesia" # features/step_definitions/common_steps.rb:633
calling as root: pidof -x -o '%PPID' gnome-terminal
call returned: [1, "", ""]
[log] CLICK on (51,16)
[log] CLICK on (58,431)
[log] CLICK on (264,470)
[log] TYPE "evince /home/amnesia/default-testpage.pdf
" 
    When I open "/home/amnesia/default-testpage.pdf" with Evince                                # features/step_definitions/evince.rb:1
    Then I see "CupsTestPage.png" after at most 20 seconds                                      # features/step_definitions/common_steps.rb:441
      FindFailed: can not find CupsTestPage.png on the screen.
      Line ?, in File ? (RuntimeError)
      ./features/step_definitions/common_steps.rb:446:in `rescue in block in <top (required)>'
      ./features/step_definitions/common_steps.rb:442:in `/^I (do not )?see "([^"]*)" after at most (\d+) seconds$/'
      features/evince.feature:18:in `Then I see "CupsTestPage.png" after at most 20 seconds'
    And I can print the current document to "/home/amnesia/output.pdf"                          # features/step_definitions/evince.rb:5
      Scenario failed at time 00:19:35

      Screenshot: https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10774-fragile-mac-address-spoofing-fails-and-macchanger-returns-false/11/artifact/build-artifacts/00:19:35_I_can_view_and_print_a_PDF_file_stored_in_non-persistent__home_amnesia.png

      Video: https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10774-fragile-mac-address-spoofing-fails-and-macchanger-returns-false/11/artifact/build-artifacts/00:19:35_I_can_view_and_print_a_PDF_file_stored_in_non-persistent__home_amnesia.mkv

Note that this is a different problem from #10775.

Associated revisions

Revision 581c58c3 (diff)
Added by intrigeri over 3 years ago

Test suite: mark one more Evince scenario as fragile.

refs: #10994

Revision 6a8b0c95 (diff)
Added by intrigeri over 3 years ago

Revert "Test suite: mark "Scenario: MAC address spoofing fails and the module is not removed" as fragile."

This reverts commit 9b174f7732d587430df026de39e193ed8ac93248.

refs: #10994

Revision fe2c086a (diff)
Added by intrigeri over 3 years ago

Revert marking of Evince print scenarios as fragile.

Refs: #10994

Revision e3cab2c7 (diff)
Added by intrigeri about 3 years ago

Test suite: mark another affected scenario as fragile (refs: #10994).

Revision 67141aa2 (diff)
Added by intrigeri about 3 years ago

Revert "Test suite: mark another affected scenario as fragile (refs: #10994)."

This reverts commit e3cab2c77cbf33674da0ee0a978ee91f38768b48.

History

#1 Updated by intrigeri over 3 years ago

  • Feature Branch set to test/10994-fragile-view-and-print-PDF

#2 Updated by intrigeri over 3 years ago

  • Description updated (diff)

#3 Updated by intrigeri over 3 years ago

  • Assignee changed from kytv to intrigeri
  • Target version changed from Tails_2.5 to Tails_2.3

I'll at least check if the problem can still be reproduced, by keeping the topic branch that re-enables these tests updated and running on Jenkins.

#4 Updated by anonym over 3 years ago

  • Target version changed from Tails_2.3 to Tails_2.4

I'll take over this one.

#5 Updated by intrigeri over 3 years ago

  • Assignee changed from intrigeri to anonym

#7 Updated by anonym about 3 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 30

This commit (currently pushed to the testing branch) might fix it:

commit a8a8992d76cc950ae79cb2b9117ad30d8f7f4b80
Author: anonym <anonym@riseup.net>
Date:   Thu May 26 18:09:45 2016 +0200

    Bump the CupsTestPage.png test suite image.

    Compared to the old image the distance between some of the elements
    (e.g. the Debian logo and color gradient) is different so let's pick
    the title only which shouldn't have such issues.

#8 Updated by anonym about 3 years ago

  • Target version changed from Tails_2.4 to Tails_2.5

#9 Updated by intrigeri about 3 years ago

  • Target version changed from Tails_2.5 to Tails_2.7

#10 Updated by intrigeri about 3 years ago

anonym wrote:

This commit (currently pushed to the testing branch) might fix it:
[...]

Sorry, it doesn't -- see attachments. Corresponding debug log:

    When I open "/home/amnesia/Persistent/default-testpage.pdf" with Evince                                       # features/step_definitions/evince.rb:1
[log] TYPE "evince /home/amnesia/Persistent/default-testpage.pdf
" 
    Then I see "CupsTestPage.png" after at most 40 seconds                                                        # features/step_definitions/common_steps.rb:459
      FindFailed: can not find CupsTestPage.png on the screen.
      Line ?, in File ? (RuntimeError)
      ./features/step_definitions/common_steps.rb:464:in `rescue in block in <top (required)>'
      ./features/step_definitions/common_steps.rb:460:in `/^I (do not )?see "([^"]*)" after at most (\d+) seconds$/'
      features/evince.feature:52:in `Then I see "CupsTestPage.png" after at most 40 seconds'
    And I can print the current document to "/home/amnesia/Persistent/output.pdf"                                 # features/step_definitions/evince.rb:5

#11 Updated by intrigeri about 3 years ago

  • Feature Branch changed from test/10994-fragile-view-and-print-PDF to wip/test/10994-fragile-view-and-print-PDF

#12 Updated by intrigeri about 3 years ago

  • Assignee deleted (anonym)
  • Target version deleted (Tails_2.7)

#13 Updated by u about 2 years ago

Dear anonym, what's the status of this ticket? Could you please clarify? Thanks!

#14 Updated by u about 2 years ago

  • Assignee set to anonym

#15 Updated by intrigeri about 2 years ago

  • Assignee deleted (anonym)

The bug is still there, and "someone should do something about it".

#16 Updated by intrigeri 6 months ago

  • Status changed from In Progress to Confirmed

Also available in: Atom PDF