Project

General

Profile

Bug #17012

"all notifications have disappeared" test suite step is fragile

Added by intrigeri 6 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
test/17012-more-robust-notification-hiding+force-all-tests
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Additional Software Packages

Description

I've seen My Additional Software list is configurable through a GUI or through notifications when I install or remove packages with APT or Synaptic fail because the I accept removing "sslh" from Additional Software step fails: no notification with a "Remove" button is visible on screen. That's because the previous notification ("Additional software installed successfully") was not hidden as planned.

As we can see in the attached video around 4:45, the click on the clock is somehow lost. That's not a big problem in itself: if the rest of the code worked fine, we would retry, and there's a good chance that this click would work eventually. However, despite this click not working and the calendar+notifications menu not hopening, for some reason, gnome_shell.child?('No Notifications', roleName: 'label') succeeds, even though there is a notification on screen, so we don't try to close notifications by clicking GnomeCloseAllNotificationsButton.png (which is nowhere to be seen anyway so that would fail, but at least, it would fail closed loudly instead of failing open and breaking the following steps).

It looks like this bug is caused by #12191 and then adding showingOnly: true to gnome_shell.child?('No Notifications', roleName: 'label') should fix it.

Here are the relevant debug log bits:

01:59:07.841611204: call returned: [3, "● tails-additional-software-install.service - Install Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-install.service; static; vendor preset: enabled)\n   Active: activating (start) since Sat 2019-08-31 09:55:50 UTC; 2min 11s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n Main PID: 9119 (python3)\n    Tasks: 3 (limit: 2349)\n   Memory: 487.2M\n   CGroup: /system.slice/tails-additional-software-install.service\n           ├─ 9119 python3 /usr/local/sbin/tails-additional-software install\n           ├─ 9324 apt-get --quiet --yes --no-remove --option DPkg::Options::=--force-confold install sslh\n           ├─32052 apt-get --quiet --yes --no-remove --option DPkg::Options::=--force-confold install sslh\n           ├─32053 sh -c /usr/local/sbin/tails-additional-software apt-post\n           └─32054 python3 /usr/local/sbin/tails-additional-software apt-post\n\nAug 31 09:56:55 amnesia tails-additional-software[9119]: [INFO] Adding system-user for sslh daemon\nAug 31 09:56:55 amnesia groupadd[9692]: group added to /etc/group: name=sslh, GID=129\nAug 31 09:56:55 amnesia groupadd[9692]: group added to /etc/gshadow: name=sslh\nAug 31 09:56:55 amnesia groupadd[9692]: new group: name=sslh, GID=129\nAug 31 09:56:55 amnesia useradd[9696]: new user: name=sslh, UID=113, GID=129, home=/nonexistent, shell=/usr/sbin/nologin\nAug 31 09:56:56 amnesia chage[9702]: changed password expiry for sslh\nAug 31 09:56:56 amnesia tails-additional-software[9119]: [INFO] Created symlink /etc/systemd/system/multi-user.target.wants/sslh.service -> /lib/systemd/system/sslh.service.\nAug 31 09:56:57 amnesia tails-additional-software[9119]: [INFO] Processing triggers for systemd (241-5) ...\nAug 31 09:56:58 amnesia tails-additional-software[9119]: [INFO] Processing triggers for man-db (2.8.5-2) ...\nAug 31 09:57:37 amnesia tails-additional-software[9119]: [INFO] Processing triggers for libc-bin (2.28-10) ...\n", ""]
01:59:17.842001895: Remote shell: calling as root: systemctl status tails-additional-software-install.service
01:59:18.582309794: call returned: [0, "● tails-additional-software-install.service - Install Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-install.service; static; vendor preset: enabled)\n   Active: active (exited) since Sat 2019-08-31 09:58:04 UTC; 7s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n  Process: 9119 ExecStart=/usr/local/sbin/tails-additional-software install (code=exited, status=0/SUCCESS)\n  Process: 32068 ExecStartPost=/usr/bin/install -m 0644 -D /dev/null /run/live-additional-software/installed (code=exited, status=0/SUCCESS)\n Main PID: 9119 (code=exited, status=0/SUCCESS)\n\nAug 31 09:57:37 amnesia tails-additional-software[9119]: [INFO] Processing triggers for libc-bin (2.28-10) ...\nAug 31 09:58:02 amnesia tails-additional-software[9119]: [INFO] [INFO] Examining package changes\nAug 31 09:58:03 amnesia tails-additional-software[9119]: [INFO] Installation completed successfully.\nAug 31 09:58:03 amnesia sudo[32059]:     root : TTY=unknown ; PWD=/ ; USER=amnesia ; ENV=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus ; COMMAND=/usr/bin/gdbus call --session --dest org.freedesktop.Notifications --object-path /org/freedesktop/Notifications --method org.freedesktop.Notifications.CloseNotification 3\nAug 31 09:58:03 amnesia sudo[32059]: pam_unix(sudo:session): session opened for user amnesia by (uid=0)\nAug 31 09:58:04 amnesia sudo[32059]: pam_unix(sudo:session): session closed for user amnesia\nAug 31 09:58:04 amnesia sudo[32063]:     root : TTY=unknown ; PWD=/ ; USER=amnesia ; COMMAND=/usr/local/lib/tails-additional-software-notify Additional software installed successfully\nAug 31 09:58:04 amnesia sudo[32063]: pam_unix(sudo:session): session opened for user amnesia by (uid=0)\nAug 31 09:58:04 amnesia sudo[32063]: pam_unix(sudo:session): session closed for user amnesia\nAug 31 09:58:04 amnesia systemd[1]: Started Install Additional Software Packages.\n", ""]
01:59:18.582802378: Remote shell: calling as root: /usr/local/sbin/tor-has-bootstrapped
01:59:18.860709954: call returned: [0, "", ""]
01:59:18.861243701: Remote shell: calling as root: test -e /run/tordate/done
01:59:19.038555127: call returned: [0, "", ""]
01:59:19.038945154: Remote shell: calling as root: test -e /run/htpdate/success
01:59:19.258225573: call returned: [0, "", ""]
01:59:19.258966797: Remote shell: calling as root: test -e '/run/live-additional-software/doomed_to_fail'
01:59:19.465074061: call returned: [1, "", ""]
01:59:19.465338362: Remote shell: calling as root: test -e '/live/persistence/TailsData_unlocked/live-additional-software.conf'
01:59:19.621160925: call returned: [0, "", ""]
01:59:19.621456144: opening file /live/persistence/TailsData_unlocked/live-additional-software.conf in 'read' mode
01:59:19.667641995: read complete
01:59:19.668033661: Remote shell: calling as root: systemctl status tails-additional-software-upgrade.service
01:59:20.199165530: call returned: [3, "● tails-additional-software-upgrade.service - Upgrade Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-upgrade.service; static; vendor preset: enabled)\n   Active: activating (start) since Sat 2019-08-31 09:58:04 UTC; 9s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n Main PID: 32069 (python3)\n    Tasks: 2 (limit: 2349)\n   Memory: 333.7M\n   CGroup: /system.slice/tails-additional-software-upgrade.service\n           ├─32069 python3 /usr/local/sbin/tails-additional-software upgrade\n           └─32070 apt-get --quiet --yes update\n\nAug 31 09:58:04 amnesia systemd[1]: Starting Upgrade Additional Software Packages...\nAug 31 09:58:04 amnesia tails-additional-software[32069]: [INFO] Starting to upgrade additional software...\nAug 31 09:58:04 amnesia tails-additional-software[32069]: [INFO] Found additional packages list.\nAug 31 09:58:04 amnesia tails-additional-software[32069]: [INFO] Saving old APT lists...\nAug 31 09:58:10 amnesia tails-additional-software[32069]: [INFO] Hit:1 tor+http://deb.tails.boum.org devel InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:2 tor+http://ftp.us.debian.org/debian buster InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:3 tor+http://ftp.us.debian.org/debian bullseye InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:4 tor+http://ftp.us.debian.org/debian sid InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:5 tor+http://deb.torproject.org/torproject.org buster InRelease\n", ""]
01:59:30.199532571: Remote shell: calling as root: systemctl status tails-additional-software-upgrade.service
01:59:30.897191575: call returned: [0, "● tails-additional-software-upgrade.service - Upgrade Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-upgrade.service; static; vendor preset: enabled)\n   Active: active (exited) since Sat 2019-08-31 09:58:20 UTC; 4s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n  Process: 32069 ExecStart=/usr/local/sbin/tails-additional-software upgrade (code=exited, status=0/SUCCESS)\n  Process: 32630 ExecStartPost=/usr/bin/install -m 0644 -D /dev/null /run/live-additional-software/upgraded (code=exited, status=0/SUCCESS)\n Main PID: 32069 (code=exited, status=0/SUCCESS)\n\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] The following package was automatically installed and is no longer required:\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO]   libpcre2-posix0\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Use 'apt autoremove' to remove it.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Installation completed successfully.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] The upgrade was successful.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Reading package lists...\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Building dependency tree...\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Reading state information...\nAug 31 09:58:20 amnesia systemd[1]: Started Upgrade Additional Software Packages.\n", ""]
01:59:30.897820851: Remote shell: calling as root: systemctl is-system-running
01:59:31.131200260: call returned: [0, "running\n", ""]
01:59:31.131617915: executing Python as amnesia: 
    import dogtail.config
    import dogtail.tree
    import dogtail.predicate
    import dogtail.rawinput
    dogtail.config.logDebugToFile = False
    dogtail.config.logDebugToStdOut = False
    dogtail.config.blinkOnActions = True
    dogtail.config.searchShowingOnly = True
    node203 = dogtail.tree.root.application('gnome-shell')
01:59:33.308370657: execution complete
01:59:33.308699187: Sikuli: calling click(L(512,10)@S(0)[0,0 1024x768])...
01:59:34.034198055: executing Python as amnesia: node204 = node203.child('No Notifications', roleName='label')
01:59:34.052903660: [log] CLICK on L(512,10)@S(0)[0,0 1024x768]
01:59:35.860216217: execution complete
01:59:35.860552193: executing Python as amnesia: node205 = node203.child('No Notifications', roleName='label')
01:59:38.200205359: execution complete
01:59:38.242693614: [log]  TYPE "#ESC." 
01:59:38.243085023: Remote shell: calling as root: test -e '/run/tails-upgrader/checked_upgrades'
01:59:38.497094785: call returned: [0, "", ""]
    And I start Tails from USB drive "__internal" and I login with persistence enabled and an administration password                                 # features/step_definitions/common_steps.rb:147

Related issues

Related to Tails - Bug #12191: Dogtail's showingOnly option is not working sometimes Confirmed 01/30/2017
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision 8ab6cc11 (diff)
Added by intrigeri 6 months ago

Test suite: make the "all notifications are disappeared" step more robust (refs: #17012)

I've seen this test fail: despite the first click on the "magic coordinates"
being lost (which is recoverable) and a notification behind displayed
on the screen, Dogtail succeeds at finding a "No Notifications" label,
which is incorrect. This looks very much like #12191 so let's apply
the usual workaround.

Revision f13ea3fd (diff)
Added by intrigeri 6 months ago

Test suite: optimize performance of the "all notifications are disappeared" (refs: #17012)

The robustness fix in the previous commit makes this step quite slow in the most
common case — that is: a notification is displayed — because we'll have to wait
for Dogtail to give up waiting on the "No Notifications" label, which is nowhere
to be seen.

Let's optimize for the case where there is at least a notification, such as the
virtual machine warning, and look for the "Clear All" [notifications]
button first.

In passing, finish Dogtail'ifying this step by dropping its remaining
Sikuli call.

Revision f71b9f7c
Added by intrigeri 6 months ago

Merge branch 'test/17012-more-robust-notification-hiding+force-all-tests' into devel (Closes: #17012)

Revision b0221497
Added by segfault 6 months ago

Merge branch 'bugfix/17023-reproducible-mimeinfo-cache' into devel (Closes: #17012)

History

#1 Updated by intrigeri 6 months ago

#2 Updated by intrigeri 6 months ago

  • Related to Bug #12191: Dogtail's showingOnly option is not working sometimes added

#3 Updated by intrigeri 6 months ago

  • Status changed from Confirmed to In Progress
  • Feature Branch set to test/17012-more-robust-notification-hiding+force-all-tests

#4 Updated by intrigeri 6 months ago

In the end, I'm also using this branch for a few unrelated test suite robustness/performance improvements.

#5 Updated by intrigeri 6 months ago

  • Status changed from In Progress to Needs Validation
  • Assignee deleted (intrigeri)

I've analyzed all failures on https://jenkins.tails.boum.org/job/test_Tails_ISO_test-17012-more-robust-notification-hiding-force-all-tests/ and noted down, in the description of each run, the reason for the failure.

I'm now confident that this branch improves things and does not introduce regressions. Please merge into devel once 3.16 and 4.0~beta2 are released :)

@hefee, maybe?

#6 Updated by hefee 6 months ago

  • Status changed from Needs Validation to In Progress
  • Assignee set to intrigeri

intrigeri wrote:

I've analyzed all failures on https://jenkins.tails.boum.org/job/test_Tails_ISO_test-17012-more-robust-notification-hiding-force-all-tests/ and noted down, in the description of each run, the reason for the failure.

I'm now confident that this branch improves things and does not introduce regressions. Please merge into devel once 3.16 and 4.0~beta2 are released :)

I have reviewed it today (8d804d51a85ab0375c08183cc4b7d24463d6dedd), they seam reasonable. Feel free to merge it, when 3.16 and 4.0~beta2 are released.

#7 Updated by intrigeri 6 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF