Project

General

Profile

Feature #10445

Feature #5288: Run the test suite automatically on autobuilt ISOs

Pass --artifacts-base-uri in Jenkins test suite wrapper

Added by bertagaz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Continuous Integration
Target version:
Start date:
10/28/2015
Due date:
% Done:

100%

QA Check:
Feature Branch:
puppet-tails:feature/10445-pass-artifacts-base-url-to-jenkins
Type of work:
Code
Blueprint:
Starter:
No
Affected tool:

Description

With #10151, a new artifacts-base-url option has been implemented, so that when a scenario fails its artifacts location are nicely printed in the logs.

We should set this option in the test suite wrapper in Jenkins as explained in #10151#note-11.


Related issues

Blocked by Tails - Feature #10151: Organize automated test suite artifacts in per-run directories Resolved 09/02/2015

History

#1 Updated by bertagaz over 3 years ago

  • Blocked by Feature #10151: Organize automated test suite artifacts in per-run directories added

#2 Updated by bertagaz over 3 years ago

  • Parent task set to #5288

#4 Updated by intrigeri over 3 years ago

  • Description updated (diff)

#5 Updated by intrigeri over 3 years ago

  • Subject changed from Configure artifacts-base-url in Jenkins test suite wrapper to Pass --artifacts-base-uri in Jenkins test suite wrapper

#6 Updated by bertagaz over 3 years ago

  • Status changed from Confirmed to In Progress
  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 80
  • QA Check set to Ready for QA
  • Feature Branch set to puppet-tails:feature/10445-pass-artifacts-base-url-to-jenkins

bertagaz wrote:

With #10151, a new artifacts-base-url option has been implemented, so that when a scenario fails its artifacts location are nicely printed in the logs.

We should set this option in the test suite wrapper in Jenkins as explained in #10151#note-11.

Merged and deployed the feature branch. A first job is already running with this option activated. But it's running fine until now... :)

#7 Updated by bertagaz over 3 years ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 80 to 100
  • QA Check deleted (Ready for QA)

A bit bold maybe, but the test job is over, and the links to capture files in the logs works, so I say myself that it works. :)

#8 Updated by intrigeri over 3 years ago

Hi,

(please drop the ticket from Cc on next reply.)

Status changed from In Progress to Resolved

A bit bold maybe, but the test job is over, and the links to capture files in the logs works, so I say myself that it works. :)

Feature #10445: Pass --artifacts-base-uri in Jenkins test suite wrapper
https://labs.riseup.net/code/issues/10445#change-48069

Thanks! Indeed I see links there:
https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/console

But these links are not where I would personally love to see them:
https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/cucumberTestReport/i2p/connecting-to-the-_i2p-irc-channel-with-the-pre-configured-account/

That is, if I want to understand a test failure, likely I'll use
the "Cucumber Test Results" page instead of the raw console output.

Dear CI team, is that easy to fix?

Cheers!

#9 Updated by bertagaz over 3 years ago

intrigeri wrote:

Thanks! Indeed I see links there:
https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/console

But these links are not where I would personally love to see them:
https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/cucumberTestReport/i2p/connecting-to-the-_i2p-irc-channel-with-the-pre-configured-account/

That is, if I want to understand a test failure, likely I'll use
the "Cucumber Test Results" page instead of the raw console output.

I think I have not been put in the email at first, but here's my 2 cents: I'm not sure that when you're looking at a failure in the log of the tests (where this links are) clicking on a link to the cucumber report page will give more information about that failure, as you already have the trace in sight. Having a direct link to the artifacts related to that failure you're looking at is maybe more relevant?

#10 Updated by intrigeri over 3 years ago

Hi,

[re-adding -ci@ to the Cc list, please drop it on next reply.]

I think I have not been put in the email at first,

See your mail from the CI team list :)

I'm not sure that when you're looking at a failure in the log of the tests (where this links are) clicking on a link to the cucumber report page will give more information about that failure, as you already have the trace in sight.

I agree.

To clarify my usecase, I was talking of looking at a failed test job from Jenkins interface, that is:

1. I'm on https://jenkins.tails.boum.org/view/RM/
2. I see a test run failed, e.g. test_Tails_ISO_stable so I hover its "Last failure" column and click "Cucumber Test Result" (since that's the nicest interface I know of to analyze such problems).
3. I land on https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_stable/lastFailedBuild/cucumberTestReport/.
4. I click on one of the failures and land on https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_stable/lastFailedBuild/cucumberTestReport/using-totem/watching-mp4-videos-stored-on-the-persistent-volume-should-work-as-expected-given-our-apparmor-confinement/

=> I don't have direct access to the artifacts.

Having a direct link to the artifacts related to that failure you're looking at is maybe more relevant?

Yes!

Also available in: Atom PDF