Bug #10288: Fix newly identified issues to make our test suite more robust and faster
Florence sometimes hides other windows, which breaks tests
Two scenarios are failing from time to time because the Florence keyboard is on top of the window where the test suite expects to see results:
cucumber features/root_access_control.feature:25 # Scenario: If no administrative password is set in Tails Greeter the live user \ should not be able to get administrative privileges through PolicyKit with the standard passwords. cucumber features/evince.feature:24 # Scenario: I cannot view a PDF file stored in non-persistent /home/amnesia/.gnupg
In this case, another scenario is also failing:
cucumber features/encryption.feature:32 # Scenario: Symmetric encryption and decryption using OpenPGP Applet
This scenarios are using snapshots, so it might be that this one is taken a bit too soon, and the Florence keyboard hasn't really finished to initialize and is still opened on the desktop.
See attached screenshots.
Test suite: hide Florence keyboard window when it doesn't vanish by itself.
#2 Updated by intrigeri about 3 years ago
- Subject changed from Florence sometimes hides other windows to Florence sometimes hides other windows, which breaks tests
I've seen it happen again in the last few days, e.g. https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10497-tor-bootstrap-is-fragile/17/. Given that this can basically affect all tests, we can't mark them all as fragile, so IMO this will need to be prioritized quite high, and may block #11355.
This problem might boil down to "
hide-on-start=true sometimes fails", i.e. a bug in Florence itself (not found any reported in the Debian or upstream BTS). But I bet this will be hard to debug, and we have other reasons (#8312, that blocks #8309, that is most plausible way of fixing #10576 some day) to get rid of Florence, so IMO a test suite -specific workaround would be good enough, i.e.: if Florence is still visible after a short while, hide it ourselves.
#3 Updated by bertagaz about 3 years ago
I've seen that too while compiling Jenkins stats. I think I mentioned it in the email I've sent on the @-ci list. I'm wondering if this happen on scenarios that are using snapshots, and when this one are made a bit too fast, while Florence is still visible, then it is interfering.
#4 Updated by intrigeri about 3 years ago
I've seen that too while compiling Jenkins stats.
... and indeed, you created this ticket :)
I'm wondering if this happen on scenarios that are using snapshots, and when this one are made a bit too fast, while Florence is still visible, then it is interfering.
Regardless of when the snapshot is taken and restored, at some point Florence's code to hide itself should run, no? It looks like the logical consequence of what you're suggesting is that this code may not work if run after restoring a snapshot. I don't see why this would be the case, but whatever, I don't mind following your lead :) What kind of solution would you suggest, following up on this tentative explanation?
#7 Updated by intrigeri about 3 years ago
- % Done changed from 0 to 10
I've pushed something, will merge it into #10497 and subtasks so I see how it works on Jenkins, on the N branches I'm monitoring closely there these days.
I'd welcome an initial very quick code review, to benefit from anonym's expertise, but don't feel pressured: this can totally wait post-2.4~rc1, and perhaps even post-2.4.