Project

General

Profile

Feature #12185

Update Debian package to run Dogtail using Python 3

Added by anonym over 2 years ago. Updated 8 days ago.

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
01/28/2017
Due date:
% Done:

0%

Feature Branch:
Type of work:
Debian
Blueprint:
Starter:
Affected tool:

Description

Currently we install python-dogtail, which uses Python 2 and makes it impossible to deal with objects containing unicode character (e.g. "Save As…" (note the triple dots)).

The Debian packaging repo for dogtail has switched to Python 3 since almost a year back, but there has been no release since then. However, I tried building a package and installing it, and it worked fine vs unicode strings at least.

Part of the "Make test suite survive tech changes" roadmap item.


Related issues

Blocks Tails - Feature #8788: Automatically test the "Tails documentation" link on the Desktop In Progress 01/24/2015
Blocked by Tails - Feature #14585: Investigate Dogtail's long-term viability Resolved 09/02/2017
Blocks Tails - Bug #15953: Make our test suite survive changes in the surrounding environment Confirmed 09/14/2018
Blocks Tails - Bug #15396: Core work Debian 2019Q1-2019Q2 Confirmed 02/08/2019

History

#1 Updated by intrigeri over 2 years ago

  • Priority changed from Normal to Low

(Indicating that this is not a blocker for 3.0. If it doesn't make it into 3.0, then it should go back to priority Normal.)

#2 Updated by intrigeri about 2 years ago

  • Priority changed from Low to Normal
  • Target version changed from Tails_3.0 to Tails_3.2

I think we'll be plenty busy with other matters during this cycle.

#3 Updated by anonym about 2 years ago

  • Subject changed from Generate Python 3 code for Dogtail to Run Dogtail using Python 3
  • Description updated (diff)

Please see the updated description. With that as background, I guess we might want to consider building our own python3-dogtail from version 0.9.9-1, until we can make Debian release it for stretch backports.

#4 Updated by spriver about 2 years ago

  • Blocks Feature #8788: Automatically test the "Tails documentation" link on the Desktop added

#5 Updated by anonym about 2 years ago

  • Blocks Feature #13239: Core work 2017Q3: Test suite maintenance added

#6 Updated by intrigeri almost 2 years ago

Looking at your 3.2 plate, I suggest you postpone this to another major release.

#7 Updated by intrigeri almost 2 years ago

  • Target version changed from Tails_3.2 to Tails_3.3

#8 Updated by intrigeri almost 2 years ago

  • Blocked by Feature #14585: Investigate Dogtail's long-term viability added

#9 Updated by intrigeri almost 2 years ago

Please check #14585 before investing lots more time into dogtail.

#10 Updated by intrigeri almost 2 years ago

  • Blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#11 Updated by intrigeri almost 2 years ago

  • Blocks deleted (Feature #13239: Core work 2017Q3: Test suite maintenance)

#12 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_3.3 to Tails_3.5

#13 Updated by intrigeri over 1 year ago

  • Target version deleted (Tails_3.5)

#14 Updated by intrigeri over 1 year ago

  • Blocks deleted (Feature #13240: Core work 2017Q4: Test suite maintenance)

#15 Updated by u over 1 year ago

If this is still needed, I suggest instead to ask Debian packagers to provide such a package. Outsource!

#16 Updated by anonym over 1 year ago

  • Assignee changed from anonym to u
  • QA Check set to Info Needed

u wrote:

If this is still needed

It is! The limitations with unicode has already required a couple ugly workarounds.

I suggest instead to ask Debian packagers to provide such a package. Outsource!

Absolutely! Can I outsource this outsourcing to you? :P Correct me if I am wrong, but my current understanding is that your role includes interfacing with Debian for things like this (?).

#17 Updated by u over 1 year ago

  • QA Check deleted (Info Needed)

Sure. I'll take care of it.

#18 Updated by u over 1 year ago

  • Target version set to Tails_3.6

#19 Updated by bertagaz over 1 year ago

  • Target version changed from Tails_3.6 to Tails_3.7

#20 Updated by bertagaz about 1 year ago

  • Target version changed from Tails_3.7 to Tails_3.8

#21 Updated by u about 1 year ago

@anonym, are you aware that Dogtail is missing a maintainer in Debian? See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777382

This not only means that we cannot ask any Debian maintainer to put out a new version, but we could theoretically commit to updating this package ourselves. As it is not actively developed, this could be a one-timer.

The tracker page currently shows no VCS information: https://tracker.debian.org/pkg/dogtail.

Upstream code seems to be living here nowadays: https://gitlab.com/dogtail/dogtail I could still ask the question there..

#22 Updated by intrigeri about 1 year ago

u wrote:

Upstream code seems to be living here nowadays: https://gitlab.com/dogtail/dogtail I could still ask the question there..

I don't think it's worth spending any more time on this before #14585 is done so feel free to reassign to anonym. It's not urgent anyway.

#23 Updated by intrigeri about 1 year ago

  • Target version changed from Tails_3.8 to Tails_3.9

#24 Updated by u 11 months ago

  • Assignee changed from u to anonym
  • Target version changed from Tails_3.9 to Tails_3.11

#26 Updated by intrigeri 11 months ago

  • Assignee changed from anonym to intrigeri

#27 Updated by intrigeri 10 months ago

  • Description updated (diff)
  • Target version changed from Tails_3.11 to 2019

#28 Updated by intrigeri 10 months ago

  • Blocks Bug #15953: Make our test suite survive changes in the surrounding environment added

#29 Updated by intrigeri 10 months ago

  • 0.9.9 was uploaded to Debian but 0.9.10 was not.
  • There's still no python3-dogtail in Debian.

#30 Updated by intrigeri 10 months ago

If we want to make this happen, at this point I think we need to start maintaining the package in Debian ourselves. At this point, that's a useful data point for #14585, not an action item.

#31 Updated by intrigeri 10 months ago

  • Assignee changed from intrigeri to u
  • Target version changed from 2019 to Tails_3.10.1
  • QA Check set to Info Needed

I've reached a conclusion on #14585 and I think we should start contributing to the maintenance of Dogtail in Debian.

First action items would be:

  1. Ensure the RC bug is forwarded upstream and its patch becomes an upstream MR.
  2. Add a python3-dogtail binary package and verify it fixes https://bugs.debian.org/728658. Once we have a WIP package I'm happy to test this in our test suite.
  3. Package upstream 0.9.10.
  4. If needed, clean up and modernize the packaging, but don't aim for perfection here and leave the package orphaned for now (it's unclear whether we'll keep using Dogtail on the long-term).

My questions for today are:

  • Would this fit in your Debian maintenance budget?
  • Any chance this is done in time for the Buster freeze?

#32 Updated by intrigeri 9 months ago

  • Target version changed from Tails_3.10.1 to Tails_3.11

#33 Updated by CyrilBrulebois 7 months ago

  • Target version changed from Tails_3.11 to Tails_3.12

#34 Updated by intrigeri 6 months ago

intrigeri wrote:

My questions for today are:

  • Would this fit in your Debian maintenance budget?

I'm still interested in knowing the answer, even if this can't happen in time for the Buster freeze: if the answer is no, then someone else should do it on the test suite maintenance budget.

  • Any chance this is done in time for the Buster freeze?

FTR this would require the package to be uploaded and accepted by ftp-masters (it has to through NEW since it adds a new binary package) by Feb 5 so it migrates to testing by Feb 11 (no new packages accepted once the soft freeze starts).
I'll be on holiday myself until Feb 4 so it's too late for me to take over this task. So if you can't do it within this timeframe, feel free to postpone as you see fit.

#35 Updated by u 6 months ago

Interestingly it seems I did not see updates of this ticket since I had reassigned it to anonym four months ago (even though it was reassigned to me the updates did go unnoticed for me). So sorry, I did also not see your question. Yes, this can fit in the budget, and I can try to make it happen before Feb 5. But if it does not work out, I'll postpone it.

Please do not hesitate to send me a personal email if you have a specific question on some kind of urgent matter next time.

#36 Updated by u 6 months ago

  • QA Check deleted (Info Needed)

#37 Updated by u 6 months ago

intrigeri wrote:

Might be fixed by https://gitlab.com/dogtail/dogtail/merge_requests/10

this looks like it was merged into master upstream.

#38 Updated by u 6 months ago

intrigeri wrote:

I've reached a conclusion on #14585 and I think we should start contributing to the maintenance of Dogtail in Debian.

First action items would be:

  1. Ensure the RC bug is forwarded upstream and its patch becomes an upstream MR.

I'm not sure what RC bug you are talking about.
Also, are you referring to this merge request https://gitlab.com/dogtail/dogtail/merge_requests/10? that was merged in the meantime?

#39 Updated by intrigeri 6 months ago

Interestingly it seems I did not see updates of this ticket since I had reassigned it to anonym four months ago. So sorry, I did also not see your question.

Ouch, I did this mistake again! I try to always first add someone as a watcher before reassigning to them (as IIRC the comment that goes with the reassignment operation is not sent to the person the ticket is reassigned to, crappy UX, yeah) and this time I forgot.

Yes, this can fit in the budget, and I can try to make it happen before Feb 5. But if it does not work out, I'll postpone it.

OK, thanks!

Please do not hesitate to send me a personal email if you have a specific question on some kind of urgent matter next time.

It's not that I would hesitate, it's rather that from my perspective, I had already put this on your radar, by assigning to you with a "Target version" not far in the future, regardless of the missed comment issue; then this was not on my own radar anymore and I simply forgot about it (I'm not managing this work and I have nothing set up to track it). But I've noticed recently that this way of putting things on someone's radar does not work well with many of our co-workers, simply because not everyone uses Redmine the same way. Point taken, I'll try to fix my assumptions.

#40 Updated by intrigeri 6 months ago

Ensure the RC bug is forwarded upstream and its patch becomes an upstream MR.

I'm not sure what RC bug you are talking about.

https://bugs.debian.org/860523 (found by looking at the bugs closed by the only upload done since I wrote that comment). It's fixed in Debian but I don't know if the fix made it upstream.

Also, are you referring to this merge request https://gitlab.com/dogtail/dogtail/merge_requests/10? that was merged in the meantime?

I have definitely referred to this MR in the context of the char encoding support problem that made us file this ticket (similar to https://bugs.debian.org/728658).
This MR may be part of the solution. It's not clear to me which one(s) of these steps is needed or sufficient to fix the problem:

  • switching to Python 3
  • applying that upstream merge request (be it as a Debian patch or by packaging an upstream release that includes it, if any)

#41 Updated by anonym 6 months ago

  • Target version changed from Tails_3.12 to Tails_3.13

#42 Updated by u 5 months ago

  • Subject changed from Run Dogtail using Python 3 to Update Debian package to run Dogtail using Python 3
  • Parent task set to #15396
  • Type of work changed from Code to Debian

#43 Updated by u 5 months ago

I'm still on it, hopefully next week.

#44 Updated by intrigeri 5 months ago

I'm still on it, hopefully next week.

tl;dr: don't worry about it, better focus on things that can still make it into Buster.

Unless I'm mistaken, it's too late to fix this in Buster: it's not possible anymore that a package goes through NEW and then migrates to testing by Feb 12 (see #12185#note-34 for details). Therefore, there's no rush anymore: as I wrote there, "if you can't do it within this timeframe, feel free to postpone as you see fit".

So I think the target is now buster-backports. This would help us get rid of some dirty hacks in our test suite. If we can have it there mid-2019, it would be awesome: we could install it in feature/buster and benefit from it in Tails 4.0 :) And if this has to wait a bit more, fine: this problem has been around for 2 years, no reason to pretend it's an emergency now that the Buster deadline is behind us.

#45 Updated by intrigeri 5 months ago

  • Parent task deleted (#15396)

#46 Updated by intrigeri 5 months ago

  • Blocks Bug #15396: Core work Debian 2019Q1-2019Q2 added

#47 Updated by u 4 months ago

  • Target version changed from Tails_3.13 to Tails_3.14

As it won't make it into Buster, I'd rater focus on more urgent things now.

#48 Updated by CyrilBrulebois about 2 months ago

  • Target version changed from Tails_3.14 to Tails_3.15

#49 Updated by CyrilBrulebois 8 days ago

  • Target version changed from Tails_3.15 to Tails_3.16

Also available in: Atom PDF