Project

General

Profile

Bug #16820

Bug #16281: Update the test suite for Buster

UEFI test is broken for Tails based on buster

Added by anonym 5 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
test/16820-uefi+force-all-tests
Type of work:
Research
Blueprint:
Starter:
Affected tool:

Description

This is about Scenario: Booting Tails from a USB drive in UEFI mode.

Both in my test VM and on Jenkins the UEFI scenario fails. We reach the part where the kernel command-line is shown, but we never manage to find TailsBootMenuKernelCmdlineUEFI.png even though the matching area is in plain sight. I've tried updating the image without success. I've tried lowering the sensitivity (0.5, which is very generous):

--- a/features/step_definitions/common_steps.rb
+++ b/features/step_definitions/common_steps.rb
@@ -239,7 +239,9 @@ Given /^Tails is at the boot menu's cmdline( after rebooting)?$/ do |reboot|
         Process.kill("CONT", tab_spammer.pid)
         dealt_with_uefi_setup = true
       end
-      @screen.wait(boot_menu_cmdline_image, 15)
+      p = Sikuli::Pattern.new(boot_menu_cmdline_image)
+      p.similar(0.5)
+      @screen.wait(p, 15)
     rescue FindFailed => e
       debug_log('We missed the boot menu before we could deal with it, ' +
                 'resetting...')

also without success. I've even @pause()@d and tried to match it, but it just doesn't work. Perhaps we're hitting some corner-case of Sikuli's image detection?

Also, if I remove the tab spammer, then the Tiano Core splash screen is shown for a shorter duration, indicating that the spammer makes makes the splash show for significantly longer: I've seen > 10x, so the boot splash never managed to appear, which is worrying. Also, pressing TAB doesn't seem to open the UEFI configuration, so we might be able to drop that workaround.


Related issues

Related to Tails - Bug #13459: Scenario "Booting Tails from a USB drive in UEFI mode" is fragile Confirmed 07/12/2017

Associated revisions

Revision c7cca007 (diff)
Added by intrigeri 5 months ago

Test suite: only send TAB every second to get the syslinux kernel command-line (refs: #16820).

Our syslinux has a timeout of 5s so sending TAB every second should be enough
to guarantee we do open the kernel command line.

As anonym reported, "the spammer makes makes the splash show for significantly
longer: I've seen > 10x, so the boot splash never managed to appear, which is
worrying".

Revision 72a7fec6 (diff)
Added by intrigeri 5 months ago

Test suite: drop workaround to make the TAB spammer compatible with the UEFI firmware (refs: #16820).

Revision 36518a63 (diff)
Added by intrigeri 5 months ago

Test suite: make image more specific (refs: #16820)

Otherwise, for some reason Sikuli can't find it, as reported
by anonym and confirmed by myself.

Revision 09efee4b (diff)
Added by intrigeri 5 months ago

Test suite: only send TAB every second to get the syslinux kernel command-line (refs: #16820).

Our syslinux has a timeout of 5s so sending TAB every second should be enough
to guarantee we do open the kernel command line.

As anonym reported, "the spammer makes makes the splash show for significantly
longer: I've seen > 10x, so the boot splash never managed to appear, which is
worrying".

Revision f7cd53ee (diff)
Added by intrigeri 5 months ago

Test suite: drop workaround to make the TAB spammer compatible with the UEFI firmware (refs: #16820).

As reported by anonym on #16820, and confirmed by my testing, pressing TAB
doesn't seem to open the UEFI configuration, so the very reason why we had this
workaround is gone.

Revision 72c4275d (diff)
Added by intrigeri 5 months ago

Test suite: make image more specific (refs: #16820)

Otherwise, for some reason Sikuli can't find it, as reported
by anonym and confirmed by myself.

Revision 288e1631
Added by intrigeri 5 months ago

Merge branch 'test/16820-uefi+force-all-tests' into feature/buster

Closes: #16820

History

#1 Updated by intrigeri 5 months ago

  • Description updated (diff)

#2 Updated by intrigeri 5 months ago

  • Related to Bug #13459: Scenario "Booting Tails from a USB drive in UEFI mode" is fragile added

#3 Updated by intrigeri 5 months ago

  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri
  • #13459#note-5 might be relevant.
  • I've tried making the tab spammer more relaxed, by increasing the sleep duration from 0.1 to 1. It makes the tianocore splash screen display only very briefly on my system, and then the kernel cmdline is there super quickly. Given our syslinux timeout is 5s, I see no reason to press tab 10 times per second. Regardless of this ticket, IMO we should raise this value to at least 0.5 and quite possibly 1s.
  • I've tried removing all the dealt_with_uefi_setup special casing but see the same behavior as anonym reported about: the kernel cmdline is there but sikuli can't find it.
  • I've tried making TailsBootMenuKernelCmdlineUEFI.png bigger (more specific than a mere "boot=live") and it fixed the problem. Will push a branch and we'll see what Jenkins thinks.

#4 Updated by intrigeri 5 months ago

  • Feature Branch set to test/16820-uefi+force-all-tests

#5 Updated by intrigeri 5 months ago

  • Parent task set to #16281

#6 Updated by intrigeri 5 months ago

Let's see what Jenkins thinks

#7 Updated by intrigeri 5 months ago

  • Status changed from In Progress to Needs Validation
  • Assignee deleted (intrigeri)

#8 Updated by hefee 5 months ago

  • Assignee set to hefee

#9 Updated by hefee 5 months ago

  • Status changed from Needs Validation to In Progress
  • Assignee changed from hefee to intrigeri

Nice to learn more about the insights of the test suite. Checked Jenkins status and looked at images changes. And as it is code removal that looks logic for me. What I don't get why we can remove the TAB spammer workaround for UEFI. Is it because we only send tab after 1sec, and after 1sec the boot splash is already passed to grub, so we don't need the workaround anymore?

#10 Updated by intrigeri 5 months ago

What I don't get why we can remove the TAB spammer workaround for UEFI.

Ouch, sorry I should of course have explained in the commit message.

The reason that as reported by anonym on this ticket, and confirmed by my testing, "pressing TAB doesn't seem to open the UEFI configuration", so the very reason why we had this workaround is gone.

#11 Updated by intrigeri 5 months ago

  • Status changed from In Progress to Needs Validation
  • Assignee changed from intrigeri to hefee

#12 Updated by hefee 5 months ago

  • Status changed from Needs Validation to In Progress
  • Assignee changed from hefee to intrigeri

intrigeri wrote:

What I don't get why we can remove the TAB spammer workaround for UEFI.

Ouch, sorry I should of course have explained in the commit message.

Yepp that would be great.

The reason that as reported by anonym on this ticket, and confirmed by my testing, "pressing TAB doesn't seem to open the UEFI configuration", so the very reason why we had this workaround is gone.

Thx for clearifing. I don't see any showstoppers anymore from merging.

#13 Updated by intrigeri 5 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

#14 Updated by intrigeri 5 months ago

Ouch, sorry I should of course have explained in the commit message.

Yepp that would be great.

Done!

Also available in: Atom PDF