Project

General

Profile

Feature #6304

Automate the most important bits of the Icedove tests

Added by bertagaz about 5 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Test suite
Target version:
Start date:
09/26/2013
Due date:
% Done:

100%

QA Check:
Pass
Feature Branch:
test/6304-icedove-tests
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:
Email Client

Description

This should be enough: set up an account with the configuration wizard, check email over IMAP and POP, send email over SMTP, all this using one server (no need to do clearnet + Onion service).


Related issues

Related to Tails - Feature #10332: Write initial Icedove tests Resolved 10/03/2015
Related to Tails - Bug #11890: Checking credentials in Icedove autoconfig wizard sometimes fails in the test suite In Progress 10/31/2016
Related to Tails - Bug #10912: Tails Installer fails to install on USB stick that has a isohybrid dd'ed to it Resolved 01/12/2016
Related to Tails - Bug #11659: Add more manual tests for Icedove Rejected 08/18/2016
Blocked by Tails - Feature #6301: Securely store secrets needed by the automated test suite Resolved 09/26/2013
Blocked by Tails - Feature #8920: Have and use a repo shared between core developers for storing test suite secrets Resolved 04/10/2015
Blocked by Tails - Feature #9493: Write Icedove manual tests for common usecases and security requirements Resolved 05/29/2015

Associated revisions

Revision 188b05a2 (diff)
Added by anonym over 2 years ago

Sanity check Icedove's autoconfigured accounts.

I.e. that IMAP is preferred and that SSL/TLS is used.

Will-fix: #6304

Revision 5255ffdb (diff)
Added by anonym over 2 years ago

Test SMTP and IMAP in Icedove.

Will-fix: #6304

Revision b804e9bc (diff)
Added by anonym over 2 years ago

Remove manual Icedove tests that now are automated.

Will-fix: #6304

Revision 695dad03
Added by intrigeri about 2 years ago

Merge remote-tracking branch 'origin/test/6304-icedove-tests' into stable

Fix-committed: #6304

History

#1 Updated by intrigeri about 5 years ago

  • Target version set to Sustainability_M1

#2 Updated by BitingBird over 4 years ago

#5663 does not have a milestone, and blocks this one - so either #5663 is aimed for 2.0, or this ticket can loose its milestone...

#3 Updated by intrigeri over 4 years ago

  • Target version deleted (Sustainability_M1)

#4 Updated by intrigeri over 4 years ago

  • Target version set to Sustainability_M1

Setting back the milestone, now that #5663 was set to 2.0.

#5 Updated by BitingBird almost 4 years ago

  • Affected tool set to Email Client

#6 Updated by anonym almost 4 years ago

  • Target version changed from Sustainability_M1 to Tails_1.8

#7 Updated by anonym almost 4 years ago

  • Blocked by deleted (Feature #6300: Framework to set up throw-away services for the test suite)

#8 Updated by anonym almost 4 years ago

  • Blocked by Feature #6301: Securely store secrets needed by the automated test suite added

#9 Updated by anonym almost 4 years ago

See #6301#note-8 for a pointer on how this should be implemented.

#10 Updated by anonym almost 4 years ago

  • Assignee set to kytv

#11 Updated by intrigeri almost 4 years ago

  • Blocked by Feature #8920: Have and use a repo shared between core developers for storing test suite secrets added

#12 Updated by intrigeri over 3 years ago

#13 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_1.8 to 246
  • Parent task changed from #6298 to #5663

#15 Updated by intrigeri over 3 years ago

  • Subject changed from Write Icedove tests to Automate the most important bits of the Icedove tests

#16 Updated by intrigeri over 3 years ago

  • Blocked by Feature #9493: Write Icedove manual tests for common usecases and security requirements added

#17 Updated by intrigeri about 3 years ago

#18 Updated by sajolida about 3 years ago

  • Target version changed from 246 to Tails_2.0

#19 Updated by u almost 3 years ago

Hi kytv,

this would be nice to have for 2.0. However, if you feel like postponing it to concentrate on a working PoC during January, please go ahead.

#20 Updated by kytv almost 3 years ago

I hope to have progress here within the next few days.

#21 Updated by u almost 3 years ago

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_2.0 to Tails_2.2

Postponing and raising priority. Everything needed to work on this ticket has been done imo.

#22 Updated by u almost 3 years ago

  • Target version changed from Tails_2.2 to Tails_2.3

#23 Updated by anonym almost 3 years ago

  • Assignee changed from kytv to anonym

#24 Updated by anonym almost 3 years ago

  • Target version changed from Tails_2.3 to Tails_2.4

#28 Updated by anonym over 2 years ago

  • Status changed from Confirmed to In Progress
  • Feature Branch set to test/6304-icedove-tests

#29 Updated by anonym over 2 years ago

  • Target version changed from Tails_2.4 to Tails_2.5

#30 Updated by intrigeri over 2 years ago

  • Target version changed from Tails_2.5 to Tails_2.7

#31 Updated by intrigeri over 2 years ago

  • Description updated (diff)

#33 Updated by anonym over 2 years ago

I have pushed something now where "most important" is:

  • Scenario: Only the expected addons are installed
  • Scenario: Enigmail is configured to use the correct keyserver
  • Scenario: Torbirdy is configured to use Tor
  • Scenario: Icedove's autoconfiguration wizard defaults to IMAP and secure protocols
  • Scenario: Icedove can send emails, and receive emails over IMAP
  • Scenario: Icedove can send emails, and receive emails over POP3

Let's see what jenkins thinks.

Note: I also wrote all the steps for testing sending/receiving mail over hidden services, but given that Chutney doesn't give us access to real world hidden services, we would have to set up our own hidden services.

#34 Updated by anonym over 2 years ago

  • % Done changed from 0 to 40

#37 Updated by anonym about 2 years ago

  • Assignee changed from anonym to bertagaz
  • % Done changed from 40 to 50
  • QA Check set to Ready for QA

It runs reliably on Jenkins from what I can tell.

#38 Updated by anonym about 2 years ago

  • Parent task deleted (#5663)

#39 Updated by bertagaz about 2 years ago

  • Assignee changed from bertagaz to anonym
  • QA Check changed from Ready for QA to Dev Needed

Sorry it took me a while to review this branch. I ran it extensively at home (over a hundred of times), and had to get it more runs in Jenkins to confirm my findings.

On the code review side I like it a lot. This feature now sounds like a perfect dogtail-ification example for other ones.

On the code testing side, I found a few glitches:

  • At home, Icedove seems to segfault quite often. It mostly happen in the step where we send an email, at different moments of this sending, but not only. Didn't test with 45.4.0 yet though. I didn't see that in Jenkins runs, so I guess we can put this aside.
  • There seem to sometimes have some problems related to at-spi: this run and this run are good example. The output of each dogtail command is then bloated with warnings that makes step parsing this output fail. Also testified this during runs at home.
  • In jenkins (never seen home), an attachment pop-up is sometimes raised when trying to send an email. I don't know where it comes from. This run and this run exhibit this problem.
  • Sometimes, probably because of network troubles, the email account login and password verification step in Icedove fails. We probably need to detect this and add a retry here. You can find such failure in this run and this run

Other than that, this branch clearly ameliorates the whole feature. Dogtail seems much more reliable.

#40 Updated by anonym about 2 years ago

  • Assignee changed from anonym to bertagaz
  • % Done changed from 50 to 60
  • QA Check changed from Dev Needed to Ready for QA

bertagaz wrote:

Sorry it took me a while to review this branch. I ran it extensively at home (over a hundred of times), and had to get it more runs in Jenkins to confirm my findings.

On the code review side I like it a lot. This feature now sounds like a perfect dogtail-ification example for other ones.

On the code testing side, I found a few glitches:

  • At home, Icedove seems to segfault quite often. It mostly happen in the step where we send an email, at different moments of this sending, but not only. Didn't test with 45.4.0 yet though. I didn't see that in Jenkins runs, so I guess we can put this aside.

Yes, it is known that Icedove recently got very segfault-prone. I don't see how we can write tests where we restart icedove if this happens without making all scenarios into single massive steps. So I don't think there's much we want to do about this, even if it turns out to be a problem. :/

  • There seem to sometimes have some problems related to at-spi: this run and this run are good example. The output of each dogtail command is then bloated with warnings that makes step parsing this output fail. Also testified this during runs at home.

See: https://bugzilla.redhat.com/show_bug.cgi?id=972257

I feel the right course of action is to file a ticket about this Dogtail and/or at-spi bug, and hope that it will be fixed in the versions shipped in Tails/Stretch. If the Icedove tests fail too often due to this on Jessie I guess we mark them @fragile (and still benefit from them for release testing) while we hopefully can have them untagged in feature/stretch. Ok?

If so, feel free to review'n'merge, but reassign it to me until I've filed this ticket. :)

  • In jenkins (never seen home), an attachment pop-up is sometimes raised when trying to send an email. I don't know where it comes from. This run and this run exhibit this problem.

If you look at the screenshots, you can see the message "Found an attachment keyword: CV". :) Let's disable this feature in the test suite. Fixed in commit de62e7a.

  • Sometimes, probably because of network troubles, the email account login and password verification step in Icedove fails. We probably need to detect this and add a retry here. You can find such failure in this run and this run

Right. According to Mozilla bug #555448 it is a catch-all error message, but we can safely assume that if we see it, it is because of Tor issue. The only problem is that Dogtail cannot handle the "status_area" widget the error is stored inside. Meh. I added some retry_tor logic that is good enough. We should have retry_tor logic at every place where we depend on the network, but I will only add it at this step, because:

  1. if we succeed here, we'll have a working circuit for the next actions too thanks to the ten minute circuit life time. Well, more or less.
  2. it's hard to add it in most other places. (Actually, one other place is easy, so I added it in commit commit:96b5118.)

Done in commit commit:756e711.

Other than that, this branch clearly ameliorates the whole feature.

Yay, that was my (secret) intention! :)

Dogtail seems much more reliable.

... but it sure has its own share of shortcomings too. :/

#41 Updated by intrigeri about 2 years ago

  • Assignee changed from bertagaz to intrigeri

The deadline we set for bertagaz' review has passed, so I'm taking this over.

#43 Updated by intrigeri about 2 years ago

  • Assignee changed from intrigeri to anonym
  • % Done changed from 60 to 70
  • QA Check changed from Ready for QA to Dev Needed

Code review passes, so the test failures I've reported above are the only blockers I can think of. Good job!

#44 Updated by anonym about 2 years ago

intrigeri wrote:

Out of the last 10 runs of icedove.feature from this branch on Jenkins:

anonym, what do you want to do about those?

All of them has this Dogtail warning of an APT-SPI error:

WARNING **: AT-SPI: Error in GetItems, sender=org.freedesktop.DBus, error=The name :1.0 was not provided by any .service files

so it is another instance of the issues mentioned in #6304#note-40 (look for the link to the upstream redhat ticket). What do you think we should do about this situation?

#45 Updated by anonym about 2 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Info Needed

#46 Updated by intrigeri about 2 years ago

  • Assignee changed from intrigeri to anonym

All of them has this Dogtail warning of an APT-SPI error:
[...]
so it is another instance of the issues mentioned in #6304#note-40 (look for the link to the upstream redhat ticket). What do you think we should do about this situation?

See: https://bugzilla.redhat.com/show_bug.cgi?id=972257

So in theory it's fixed in dogtail 0.9.1 (testing/sid) but not in 0.9.0 (Jessie).

I feel the right course of action is to file a ticket about this Dogtail and/or at-spi bug, and hope that it will be fixed in the versions shipped in Tails/Stretch. If the Icedove tests fail too often due to this on Jessie I guess we mark them @fragile (and still benefit from them for release testing) while we hopefully can have them untagged in feature/stretch. Ok?

Indeed, a 20% failure rate feels too high not to mark these tests as fragile. I could agree to merge this branch with the affected tests marked as fragile, but first I'd like to have a better understanding of what a real solution could be (without any occurrence of "hope" in the description of the solution ;)

But, wait: can't we install python-dogtail from testing, that supposedly fixes the problem, in our ISO? I've just upgraded the package manually in a running Tails and at least APT-wise it went just fine.

#47 Updated by anonym about 2 years ago

  • QA Check changed from Info Needed to Dev Needed

intrigeri wrote:

All of them has this Dogtail warning of an APT-SPI error:
[...]
so it is another instance of the issues mentioned in #6304#note-40 (look for the link to the upstream redhat ticket). What do you think we should do about this situation?

See: https://bugzilla.redhat.com/show_bug.cgi?id=972257

So in theory it's fixed in dogtail 0.9.1 (testing/sid) but not in 0.9.0 (Jessie).

Yes. Or maybe the atspi2 version in Stretch works better.

I feel the right course of action is to file a ticket about this Dogtail and/or at-spi bug, and hope that it will be fixed in the versions shipped in Tails/Stretch. If the Icedove tests fail too often due to this on Jessie I guess we mark them @fragile (and still benefit from them for release testing) while we hopefully can have them untagged in feature/stretch. Ok?

Indeed, a 20% failure rate feels too high not to mark these tests as fragile. I could agree to merge this branch with the affected tests marked as fragile, but first I'd like to have a better understanding of what a real solution could be (without any occurrence of "hope" in the description of the solution ;)

Well, I'm a bit lost, since I have no clue how atspi2 works, and that's where the bug is. So, I suppose fixing the bug in atspi2 is the real solution.

But, wait: can't we install python-dogtail from testing, that supposedly fixes the problem, in our ISO? I've just upgraded the package manually in a running Tails and at least APT-wise it went just fine.

I pushed a commit doing that, so let's see what happens for 10 more runs.

#48 Updated by intrigeri about 2 years ago

So in theory it's fixed in dogtail 0.9.1 (testing/sid) but not in 0.9.0 (Jessie).

Yes. Or maybe the atspi2 version in Stretch works better.

Just to clarify, I meant that https://bugzilla.redhat.com/show_bug.cgi?id=972257 is marked as resolved in 0.9.1 upstream.

Well, I'm a bit lost, since I have no clue how atspi2 works, and that's where the bug is. So, I suppose fixing the bug in atspi2 is the real solution.

… and as written above, it seems that this has already been done 2 years ago :)

But, wait: can't we install python-dogtail from testing, that supposedly fixes the problem, in our ISO? I've just upgraded the package manually in a running Tails and at least APT-wise it went just fine.

I pushed a commit doing that, so let's see what happens for 10 more runs.

Cool!

#49 Updated by bertagaz about 2 years ago

Yay, I missed what happened in this ticket, I was busy outside, but meanwhile I've still run this branch locally and in Jenkins, and analyzed the errors I've seen.

First, this branch greatly enhanced the runs. The failure ratio has lowered a lot more.

As you noted one big remaining issue is the dogtail/at-spi failures. If the upgrades you're trying to test is not solving the problem, we may still workaround the issue: what I've seen is that if this error happens, it always does at the beginning of the feature, and then is spread to all the scenarios thanks to our snapshot system.

So maybe we can add a step right at the beginning of the session opening, after the "available upgrades have been checked" that check if dogtail works fine, and restart the snapshot process if it isn't. But well, let's hope that the simpler upgrading at-spi and dogtail works. If we need to use this trick, I think we should open a dedicated ticket and merge this branch.

The only other issue I've seen when Icedove is checking the credentials in the wizard. Even retry_tor() it doesn't prevent some failures from time to time. One thing we may do is just ignore this failures: in this step, what we need to know is that there's no network traffic outside of Tor, not that the login we know works does. So we may just try it in the related scenario, ensure there were traffic outside of Tails and only through Tor no matter what the result of this credentials check is, and then in the other scenario simply drop a configuration file in icedove directory. But maybe that's not required, or pertains to another ticket.

On the code side, I don't like so much this:

     When I start Icedove
     And Icedove has started

I think we could just write "When I start Icedove".

That's the only remaining troubles I've seen (apart from one segfault out of 40 local runs). We're getting close !

Now, I'll switch to #11874 :)

#50 Updated by anonym about 2 years ago

anonym wrote:

intrigeri wrote:

But, wait: can't we install python-dogtail from testing, that supposedly fixes the problem, in our ISO? I've just upgraded the package manually in a running Tails and at least APT-wise it went just fine.

I pushed a commit doing that, so let's see what happens for 10 more runs.

After this I've seen two runs where the error is logged, but Dogtail doesn't fail:

I guess that is a promising sign!

#51 Updated by anonym about 2 years ago

anonym wrote:

After this I've seen two runs where the error is logged, but Dogtail doesn't fail:

Here's a third:

Now we're up in nine runs, and 3/9 is pretty consistent with the previous 20% error rate reported by intrigeri (#6304#note-46) given the sample size, but it gives confidence that this new Dogtail version catches these errors and prevents the failure. What a relief! Over the past months I've started building up a feeling that Dogtail has subtle issues like this one that makes it less robust than what we require, and perhaps worse (accuracy-wise) than sikuli. Now my confidence is returning! :)

bertagaz, I'll look at your comment #6304#note-49 soon, but not today.

#52 Updated by anonym about 2 years ago

bertagaz wrote:

As you noted one big remaining issue is the dogtail/at-spi failures. If the upgrades you're trying to test is not solving the problem, we may still workaround the issue: what I've seen is that if this error happens, it always does at the beginning of the feature, and then is spread to all the scenarios thanks to our snapshot system.

So maybe we can add a step right at the beginning of the session opening, after the "available upgrades have been checked" that check if dogtail works fine, and restart the snapshot process if it isn't. But well, let's hope that the simpler upgrading at-spi and dogtail works. If we need to use this trick, I think we should open a dedicated ticket and merge this branch.

I guess we don't need this now.

The only other issue I've seen when Icedove is checking the credentials in the wizard. Even retry_tor() it doesn't prevent some failures from time to time.

For the record I've never seen this locally or on Jenkins. I admittedly have not looked at all Jenkins runs -- have you seen it there?

One thing we may do is just ignore this failures: in this step, what we need to know is that there's no network traffic outside of Tor, not that the login we know works does. So we may just try it in the related scenario, ensure there were traffic outside of Tails and only through Tor no matter what the result of this credentials check is, and then in the other scenario simply drop a configuration file in icedove directory. But maybe that's not required, or pertains to another ticket.

The credentials checking is part of the account creation process, which must succeed for the rest of the scenarios steps, so this must be robust any way. I would not be surprised if there's a bug in the JavaScript mess that is the autoconfiguration wizard that prevents it from recovering (to a state where it can retest the credentials), hence making our try_tor magic not work. I'm gonna look into reproducing this. Do you happen to have a debug log of when this happens?

Still, I propose that if the Jenkins results continue to look this good we give merging it (without @fragile tags) a try, and possibly revisit this if it turns out to be a problem.

On the code side, I don't like so much this:

[...]

I think we could just write "When I start Icedove".

Fixed!

That's the only remaining troubles I've seen (apart from one segfault out of 40 local runs).

So far I've seen no segfault on Jenkins, by the way.

Thanks for the review and test data!


Everything's been looking good on Jenkins, except an error in this log:

Note that it has the AT-SPI error but handles it nicely. The error I am talking about is:

calling as amnesia: echo '#!/usr/bin/python
from dogtail import tree
from dogtail.config import config
config.searchShowingOnly = True
application = tree.root.application('"'"'Icedove'"'"')
from dogtail import predicate
node = None
for n in application.child('"'"'Add-ons Manager - Icedove Mail/News'"'"', roleName='"'"'frame'"'"').child('"'"'amnesia branding'"'"', roleName='"'"'label'"'"').parent.parent.findChildren(predicate.GenericPredicate()):
    if str(n.path) == '"'"'/org/a11y/atspi/accessible/228'"'"':
        node = n
        break
assert(node)
print(node.name)' >> '/tmp/tmp.nvQs511obP'
call returned: [1744, 0, "", ""]
calling as amnesia: /usr/bin/python '/tmp/tmp.nvQs511obP'
call returned: [1745, 0, "Creating logfile at /tmp/dogtail-amnesia/logs/tmp.nvQs511obP_20161028-004613_debug ...\nDogtail: warning: application may be hanging\nTorBirdy This extension configures Thunderbird to make connec\u2026 More Release Notes: Loading\u2026 Preferences Disable\n", "\n** (tmp.nvQs511obP:5663): WARNING **: AT-SPI: Error in GetItems, sender=org.freedesktop.DBus, error=The name :1.0 was not provided by any .service files\n"]
calling as root: rm -f '/tmp/tmp.nvQs511obP'
call returned: [1746, 0, "", ""]
    Then I see that only the amnesia branding, Enigmail and TorBirdy addons are enabled in Icedove # features/step_definitions/icedove.rb:59
      <nil> expected to not be nil. (Test::Unit::AssertionFailedError)

The interesting part is the remote shell stdout result that starts with "Creating logfile" and then has "TorBirdy This extension [...]". Normally we'd only get this latter part, and that's what we expect, and why we get an assertion failure here. We need to make sure that Dogtail never writes to stdout unless we tell it to, since we depend on the exact stdout ouput many times. I'll look into this.

#53 Updated by anonym about 2 years ago

anonym wrote:

We need to make sure that Dogtail never writes to stdout unless we tell it to, since we depend on the exact stdout ouput many times. I'll look into this.

Fixed in 1eb8566.

Also, 55cdd5f might be interesting.

I'm waiting for results on Jenkins before I put this one back for review.

#54 Updated by intrigeri about 2 years ago

Still, I propose that if the Jenkins results continue to look this good we give merging it (without @fragile tags) a try, and possibly revisit this if it turns out to be a problem.

ACK

#55 Updated by bertagaz about 2 years ago

anonym wrote:

So maybe we can add a step right at the beginning of the session opening, after the "available upgrades have been checked" that check if dogtail works fine, and restart the snapshot process if it isn't. But well, let's hope that the simpler upgrading at-spi and dogtail works. If we need to use this trick, I think we should open a dedicated ticket and merge this branch.

I guess we don't need this now.

Seems so.

The only other issue I've seen when Icedove is checking the credentials in the wizard. Even retry_tor() it doesn't prevent some failures from time to time.

For the record I've never seen this locally or on Jenkins. I admittedly have not looked at all Jenkins runs -- have you seen it there?

I usually have more network transient errors than in Jenkins, so that's probably why. I don't remember having seen that in Jenkins, specially since your last commits with the retry_tor wrapping (and I've look at most of the last runs since then). I'll have a quick look just in case, but I think it works better on lizard than $HOME. My connectivity is probably a bot more shitty.

The credentials checking is part of the account creation process, which must succeed for the rest of the scenarios steps, so this must be robust any way. I would not be surprised if there's a bug in the JavaScript mess that is the autoconfiguration wizard that prevents it from recovering (to a state where it can retest the credentials), hence making our try_tor magic not work. I'm gonna look into reproducing this. Do you happen to have a debug log of when this happens?

Yeah I think I can find one or two. I'll dig and post later.

Still, I propose that if the Jenkins results continue to look this good we give merging it (without @fragile tags) a try, and possibly revisit this if it turns out to be a problem.

I agree, we'll see in the November fragile tag round if that's a problem.

That's the only remaining troubles I've seen (apart from one segfault out of 40 local runs).

So far I've seen no segfault on Jenkins, by the way.

Me neither. That's good news. :)

Thanks for the review and test data!

Thanks for the code! :)

#56 Updated by anonym about 2 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

anonym wrote:

anonym wrote:

We need to make sure that Dogtail never writes to stdout unless we tell it to, since we depend on the exact stdout ouput many times. I'll look into this.

Fixed in 1eb8566.

Also, 55cdd5f might be interesting.

I'm waiting for results on Jenkins before I put this one back for review.

4/4 Jenkins runs with the new commits has finished without any Icedove or Dogtail issues. Review'n'merge, please!

bertagaz wrote:

anonym wrote:

The only other issue I've seen when Icedove is checking the credentials in the wizard. Even retry_tor() it doesn't prevent some failures from time to time.

For the record I've never seen this locally or on Jenkins. I admittedly have not looked at all Jenkins runs -- have you seen it there?

I usually have more network transient errors than in Jenkins, so that's probably why. I don't remember having seen that in Jenkins, specially since your last commits with the retry_tor wrapping (and I've look at most of the last runs since then). I'll have a quick look just in case, but I think it works better on lizard than $HOME. My connectivity is probably a bot more shitty.

Ok. Let's open a ticket about it, but only if you manage to get some useful log to attach to it -- otherwise I have nothing to try to understand the problem.

The credentials checking is part of the account creation process, which must succeed for the rest of the scenarios steps, so this must be robust any way. I would not be surprised if there's a bug in the JavaScript mess that is the autoconfiguration wizard that prevents it from recovering (to a state where it can retest the credentials), hence making our try_tor magic not work. I'm gonna look into reproducing this. Do you happen to have a debug log of when this happens?

Yeah I think I can find one or two. I'll dig and post later.

... to the above mentioned new ticket, please!

#57 Updated by intrigeri about 2 years ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 70 to 100

#58 Updated by intrigeri about 2 years ago

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#59 Updated by intrigeri about 2 years ago

  • Related to Bug #11890: Checking credentials in Icedove autoconfig wizard sometimes fails in the test suite added

#60 Updated by intrigeri about 2 years ago

anonym wrote:

bertagaz wrote:

anonym wrote:

The only other issue I've seen when Icedove is checking the credentials in the wizard. Even retry_tor() it doesn't prevent some failures from time to time.

For the record I've never seen this locally or on Jenkins. I admittedly have not looked at all Jenkins runs -- have you seen it there?

I usually have more network transient errors than in Jenkins, so that's probably why. I don't remember having seen that in Jenkins, specially since your last commits with the retry_tor wrapping (and I've look at most of the last runs since then). I'll have a quick look just in case, but I think it works better on lizard than $HOME. My connectivity is probably a bot more shitty.

Ok. Let's open a ticket about it, but only if you manage to get some useful log to attach to it -- otherwise I have nothing to try to understand the problem.

The credentials checking is part of the account creation process, which must succeed for the rest of the scenarios steps, so this must be robust any way. I would not be surprised if there's a bug in the JavaScript mess that is the autoconfiguration wizard that prevents it from recovering (to a state where it can retest the credentials), hence making our try_tor magic not work. I'm gonna look into reproducing this. Do you happen to have a debug log of when this happens?

Yeah I think I can find one or two. I'll dig and post later.

... to the above mentioned new ticket, please!

#11890

#61 Updated by bertagaz about 2 years ago

  • Status changed from Fix committed to Resolved

#62 Updated by intrigeri about 2 years ago

  • Related to Bug #10912: Tails Installer fails to install on USB stick that has a isohybrid dd'ed to it added

#63 Updated by intrigeri about 2 years ago

  • Related to Bug #11659: Add more manual tests for Icedove added

Also available in: Atom PDF