Project

General

Profile

Feature #14585

Feature #12213: Wayland in Tails 5.0 (Bullseye)

Investigate Dogtail's long-term viability

Added by intrigeri over 1 year ago. Updated 2 months ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Test suite
Target version:
Start date:
09/02/2017
Due date:
% Done:

100%

QA Check:
Feature Branch:
Type of work:
Communicate
Blueprint:
Starter:
Affected tool:

Description

Part of the "Make test suite survive tech changes" roadmap item.

We have multiple concerns about the long-term viability of using Dogtail:

  • intrigeri heard a GUADEC speaker mention that dogtail does not work on Wayland. If that's indeed the case, then we should look into this problem before we keep porting tons of tests to dogtail: it would be a waste of time to do so only to have to port them to another technology again a year later. See https://gitlab.com/dogtail/dogtail/issues/3.
  • The package is orphaned in Debian and was of course not updated to the latest upstream release.
  • Fedora (and Redhat in general?) now uses openQA, which only supports image-based testing.
  • Dogtail development has slowed down considerably since we adopted it. Here's the a number of commits per year breakdown:
    2011: 4
    2012: 52
    2013: 25
    2014: 27
    2015: 13
    2016: 41
    2017: 9

Related issues

Related to Tails - Bug #12191: Dogtail's showingOnly option is not working sometimes Confirmed 01/30/2017
Blocks Tails - Feature #12185: Run Dogtail using Python 3 Confirmed 01/28/2017
Blocks Tails - Bug #15953: Make our test suite survive changes in the surrounding environment Confirmed 09/14/2018

History

#1 Updated by intrigeri about 1 year ago

#2 Updated by anonym about 1 year ago

  • Type of work changed from Code to Communicate

Fuck. I'll try to get a clarification about dogtail's status from the upstream (it seems they've moved from Fedora's infra to GitLab), and what the plans for Wayland are. Until then I'll suspend all my Dogtail work. :_(

#3 Updated by anonym about 1 year ago

I've sent an email to its mailing list asking for clarification. We'll see.

Also, Dogtail 0.9.10 was released this May, which inspires some confidence!

#4 Updated by anonym about 1 year ago

  • Subject changed from Make our dogtail-based automated tests work on Wayland to Investigate Dogtail's long-term viability
  • Description updated (diff)

#5 Updated by anonym about 1 year ago

  • Blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#6 Updated by anonym about 1 year ago

  • Status changed from Confirmed to In Progress
  • Priority changed from Normal to Elevated
  • Target version set to Tails_3.3
  • % Done changed from 0 to 10

#7 Updated by intrigeri about 1 year ago

  • Target version changed from Tails_3.3 to Tails_3.5

#8 Updated by intrigeri about 1 year ago

  • Target version deleted (Tails_3.5)

#9 Updated by intrigeri 12 months ago

  • Blocks deleted (Feature #13240: Core work 2017Q4: Test suite maintenance)

#10 Updated by intrigeri 12 months ago

  • Target version set to 2019

#11 Updated by u 7 months ago

anonym wrote:

I've sent an email to its mailing list asking for clarification. We'll see.

I can't find your email on the list archives. It looks like they've ceased to exist since 10 months already. I wanted to see if there were any replies to this.

#12 Updated by intrigeri 7 months ago

  • Description updated (diff)

#13 Updated by intrigeri 5 months ago

  • intrigeri heard a GUADEC speaker mention that dogtail does not work on Wayland.

I see that dogtail uses pyatspi to generate mouse and keyboard events so I've followed the suggestions on https://www.freedesktop.org/wiki/Accessibility/AT-SPI2/ and did that:

export GTK_MODULES=gail:atk-bridge
export OOO_FORCE_DESKTOP=gnome
export GNOME_ACCESSIBILITY=1
export QT_ACCESSIBILITY=1

and gsettings set org.gnome.desktop.interface toolkit-accessibility true which is a pre-requisite anyway.

I've tried on my own GNOME + Wayland session (sid), that has XWayland, and while dogtail can definitely query info about GUIs, I could not manage to make it interact with gedit (that seems to run as a native Wayland app): Dogtail thinks it has successfully clicked around but nothing visible happens in the app. So indeed, dogtail is currently broken for native Wayland apps.

import dogtail
import dogtail.tree
dogtail.tree.root.application('gedit').window('Untitled Document 1 - gedit').button('Save').click()

Now, if I start gedit with GDK_BACKEND=x11 gedit (XWayland) then things work again as in the good old days… without the security benefits of Wayland. But that gives us one potential option: we could use the x11 backend in the test suite for the apps we want to test with Dogtail, but not in a real Tails. The main problem with this option is of course that we might miss some Wayland-specific issues. I would have been concerned with that a couple years ago but it's been a year or so since the last time I noticed such a problem on my own system: things have been fixed.

#14 Updated by intrigeri 5 months ago

  • Description updated (diff)

#15 Updated by u 4 months ago

  • Related to Bug #12191: Dogtail's showingOnly option is not working sometimes added

#16 Updated by intrigeri 4 months ago

  • Assignee changed from anonym to intrigeri
  • Target version changed from 2019 to Tails_3.10.1

We should make a decision wrt. dogtail before the work to port the test suite to Buster resumes. I'll handle it.

#17 Updated by nodens 4 months ago

#18 Updated by intrigeri 3 months ago

  • Description updated (diff)

#19 Updated by intrigeri 3 months ago

  • Blocks Bug #15953: Make our test suite survive changes in the surrounding environment added

#20 Updated by intrigeri 3 months ago

#21 Updated by intrigeri 3 months ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 10 to 100

tl;dr: let's invest as little as we can into Dogtail but for now, we can use it.

It's hard to tell about long-term but so far:

  • It does not seem unreasonable to keep using Dogtail and even to port specific tests to it when it gives us substantial benefits. But let's not port tests to Dogtail just for fun (anyway, as I've learned the hard way when writing the VeraCrypt tests, Dogtail does not deliver the expected robustness, and sometimes Sikuli works much better).
  • A workaround is available for the lack of Wayland support (which is really not a problem in Dogtail itself, but rather missing a11y APIs in Wayland).

Now, we should start contributing to the maintenance of the package in Debian, so we get Python 3 and Unicode support; I'll report this on #12185.

#22 Updated by intrigeri 2 months ago

  • Target version changed from Tails_3.10.1 to Tails_3.9.1

Also available in: Atom PDF