Project

General

Profile

Bug #10474

Bug #10288: Fix newly identified issues to make our test suite more robust and faster

Scenario "Connecting to the #i2p IRC channel" is fragile

Added by bertagaz about 4 years ago. Updated 11 months ago.

Status:
Rejected
Priority:
Low
Assignee:
-
Category:
Test suite
Target version:
-
Start date:
11/04/2015
Due date:
% Done:

30%

Feature Branch:
kytv/test/10474-i2p-irc-is-fragile
Type of work:
Code
Blueprint:
Starter:
Affected tool:
I2P

Description

As one can see in this job in Jenkins and others, this scenario fails often and seems a bit fragile:

3:26:19     Then Pidgin successfully connects to the "I2P" account                                          # features/step_definitions/pidgin.rb:319
03:26:19       FindFailed: can not find PidginAccount_I2P.png on the screen.
03:26:19       Line ?, in File ? (RuntimeError)
03:26:19       ./features/step_definitions/pidgin.rb:313:in `deactivate_and_activate_pidgin_account'
03:26:19       ./features/step_definitions/pidgin.rb:326:in `block (2 levels) in <top (required)>'
03:26:19       ./features/support/helpers/misc_helpers.rb:128:in `call'
03:26:19       ./features/support/helpers/misc_helpers.rb:128:in `rescue in block in retry_action'
03:26:19       ./features/support/helpers/misc_helpers.rb:120:in `block in retry_action'
03:26:19       ./features/support/helpers/misc_helpers.rb:119:in `loop'
03:26:19       ./features/support/helpers/misc_helpers.rb:119:in `retry_action'
03:26:19       ./features/support/helpers/misc_helpers.rb:109:in `retry_i2p'
03:26:19       ./features/step_definitions/pidgin.rb:330:in `call'
03:26:19       ./features/step_definitions/pidgin.rb:330:in `/^Pidgin successfully connects to the "([^"]+)" account$/'

Related issues

Related to Tails - Bug #10506: Opening Pidgin's Account Manager in the test suite is fragile Resolved 11/06/2015

History

#3 Updated by intrigeri about 4 years ago

  • Status changed from New to Confirmed

#4 Updated by intrigeri about 4 years ago

  • Affected tool set to I2P

#6 Updated by kytv about 4 years ago

  • Target version changed from Tails_1.8 to 246

#7 Updated by kytv about 4 years ago

  • Feature Branch set to kytv/test/10474-i2p-irc-is-fragile

#8 Updated by kytv about 4 years ago

This one is interesting. It failed in deactivate_and_activate_pidgin_account.

calling as amnesia: xdotool search --name 'Buddy List' windowactivate --sync
call returned: [0, "", ""]
I2P operation failed (Try 1 of 15) with:
FindAnyFailed: can not find any of the images ["PidginI2PChannelEntry.png", "PidginReconnect.png"] on the screen
Deactivating and reactivating Pidgin account I2P
[log] Ctrl+TYPE "a" 
[log] CLICK on (446,309)
[log] TYPE " " 
[log] CLICK on (711,509)
[log] Ctrl+TYPE "a" 
    Then Pidgin successfully connects to the "I2P" account                                          # features/step_definitions/pidgin.rb:319
      FindFailed: can not find PidginAccount_I2P.png on the screen.
      Line ?, in File ? (RuntimeError)
      ./features/step_definitions/pidgin.rb:313:in `deactivate_and_activate_pidgin_account'
      ./features/step_definitions/pidgin.rb:326:in `block (2 levels) in <top (required)>'
      ./features/support/helpers/misc_helpers.rb:128:in `call'
      ./features/support/helpers/misc_helpers.rb:128:in `rescue in block in retry_action'
      ./features/support/helpers/misc_helpers.rb:120:in `block in retry_action'
      ./features/support/helpers/misc_helpers.rb:119:in `loop'
      ./features/support/helpers/misc_helpers.rb:119:in `retry_action'
      ./features/support/helpers/misc_helpers.rb:109:in `retry_i2p'
      ./features/step_definitions/pidgin.rb:330:in `call'
      ./features/step_definitions/pidgin.rb:330:in `/^Pidgin successfully connects to the "([^"]+)" account$/'
      features/i2p.feature:62:in `Then Pidgin successfully connects to the "I2P" account'
    And I can join the "#i2p" channel on "I2P"                                                      # features/step_definitions/pidgin.rb:361
      Scenario failed at time 01:52:47

It failed to connect within a minute. I wrote the test so that it will deactivate the account then reactivate it. During this test it only managed to deactivate the account. This step includes

def deactivate_and_activate_pidgin_account(account)
  debug_log("Deactivating and reactivating Pidgin account #{account}")
  step "I open Pidgin's account manager window" 
  step "I deactivate the \"#{account}\" Pidgin account" 
  step "I close Pidgin's account manager window" 
  step "I open Pidgin's account manager window" 
  step "I activate the \"#{account}\" Pidgin account" 
  step "I close Pidgin's account manager window" 
end

The job referenced in the description failed for the same reason. Hmmmmm. So what failed specifically was


  step "I open Pidgin's account manager window" 
  step "I activate the \"#{account}\" Pidgin account" 
  step "I close Pidgin's account manager window" 

Hmm :/

#9 Updated by kytv about 4 years ago

It may not be the only problem with this scenario but we can (probably) cheaply improve robustness in all of the Pidgin tests which require connecting to an account by opening the Account manager using the menus.

#10 Updated by kytv about 4 years ago

  • Related to Bug #10506: Opening Pidgin's Account Manager in the test suite is fragile added

#11 Updated by sajolida about 4 years ago

  • Target version changed from 246 to Tails_2.0

#12 Updated by kytv about 4 years ago

Should probably update the image to look for Topic for #i2p

#13 Updated by intrigeri about 4 years ago

  • Target version changed from Tails_2.0 to Tails_2.2

(We're going to mark as fragile all tests that depend on Tor to have bootstrapped for the moment => not so urgent.)

#14 Updated by kytv about 4 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 30

I think this will be mostly solved when the "drive-by" changes for #10684 are merged.

#15 Updated by intrigeri almost 4 years ago

We can't possibly fix all robustness and performance issues we discover as part of M6 => prioritizing areas that impact most users.

#16 Updated by anonym almost 4 years ago

  • Target version changed from Tails_2.2 to Tails_2.3

#17 Updated by anonym over 3 years ago

  • Target version changed from Tails_2.3 to Tails_2.4

#18 Updated by intrigeri over 3 years ago

Seen it again (will attach video):

    When I start Pidgin through the GNOME menu                                                      # features/step_definitions/pidgin.rb:291
    Then I see Pidgin's account manager window                                                      # features/step_definitions/pidgin.rb:301
[log] CLICK on (438,332)
[log] TYPE " " 
    When I activate the "I2P" Pidgin account                                                        # features/step_definitions/pidgin.rb:309
[log] CLICK on (710,528)
    And I close Pidgin's account manager window                                                     # features/step_definitions/pidgin.rb:305
calling as amnesia: xdotool search --name 'Buddy List' windowactivate --sync
call returned: [0, "", ""]
I2P operation failed (Try 1 of 15) with:
FindAnyFailed: can not find any of the images ["PidginI2PChannelEntry.png", "PidginReconnect.png"] on the screen
Deactivating and reactivating Pidgin account I2P
[log] CLICK on (257,186)
[log] CLICK on (300,214)
[log] CLICK on (438,332)
[log] TYPE " " 
[log] CLICK on (710,528)
[log] CLICK on (257,186)
[log] CLICK on (300,214)
[log] CLICK on (438,332)
[log] TYPE " " 
[log] CLICK on (710,528)
calling as amnesia: xdotool search --name 'Buddy List' windowactivate --sync
call returned: [0, "", ""]
    Then Pidgin successfully connects to the "I2P" account                                          # features/step_definitions/pidgin.rb:334
[log] DOUBLE CLICK on (192,245)
calling as amnesia: xdotool search --name 'InfoServ' windowactivate --sync
call returned: [0, "", ""]
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
[log] Ctrl+TYPE "w" 
    And I can join the pre-configured "#i2p" channel on "I2P"                                       # features/step_definitions/pidgin.rb:376
      try_for() timeout expired
      Last ignored exception was: RuntimeError: FindFailed: can not find PidginI2PConversationTab.png on the screen.
      Line ?, in File ? (Timeout::Error)
      ./features/support/helpers/misc_helpers.rb:83:in `rescue in try_for'
      ./features/support/helpers/misc_helpers.rb:33:in `try_for'
      ./features/step_definitions/pidgin.rb:392:in `/^I can join the( pre-configured)? "([^"]+)" channel on "([^"]+)"$/'
      features/i2p.feature:67:in `And I can join the pre-configured "#i2p" channel on "I2P"'
      Scenario failed at time 00:49:21

      Screenshot: https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10376-fix-startup-page-roadmap-test-is-fragile/7/artifact/build-artifacts/00:49:21_Connecting_to_the__i2p_IRC_channel_with_the_pre-configured_account.png

      Video: https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10376-fix-startup-page-roadmap-test-is-fragile/7/artifact/build-artifacts/00:49:21_Connecting_to_the__i2p_IRC_channel_with_the_pre-configured_account.mkv

I suspect that the #i2p tab was open but hidden (due to the infoserv + nickserv tabs), so it was closed with ctrl-w. Maybe we should make the chat window fullscreen to avoid such problems? Will mark this test as fragile, anyway.

#20 Updated by intrigeri over 3 years ago

  • Assignee deleted (kytv)
  • Target version deleted (Tails_2.4)

#21 Updated by spriver over 2 years ago

#22 Updated by u over 2 years ago

  • Priority changed from Normal to Low

Lowering priority as we currently don't ship i2p.

#23 Updated by u about 2 years ago

This can't block #8668 because we don't even ship i2p anymore.

#25 Updated by intrigeri 11 months ago

#26 Updated by intrigeri 11 months ago

  • Status changed from In Progress to Rejected

Let's reopen this once there are people working on automatic testing of I2P in Tails.

Also available in: Atom PDF