Project

General

Profile

Feature #8165

Make our automated test suite run in a Jessie environment

Added by intrigeri almost 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
Test suite
Target version:
Start date:
10/20/2014
Due date:
% Done:

100%

Feature Branch:
test/8165-run-test-suite-in-jessie
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Related issues

Blocked by Tails - Feature #7212: Try running our automated test suite in a Jessie environment Resolved 05/10/2014
Blocks Tails - Feature #5683: Puppet module to deploy systems able to run our test suite Resolved
Blocked by Tails - Bug #9074: python-xmpp broken in Jessie which breaks otr-bot Resolved 03/18/2015

Associated revisions

Revision c035baf3
Added by intrigeri over 4 years ago

Merge branch 'test/8165-run-test-suite-in-jessie' into stable

Fix-committed: #8165

History

#1 Updated by intrigeri almost 5 years ago

  • Blocked by Feature #7212: Try running our automated test suite in a Jessie environment added

#2 Updated by intrigeri almost 5 years ago

  • Subject changed from Make our automated test suite in a Jessie environment to Make our automated test suite run in a Jessie environment

#3 Updated by anonym almost 5 years ago

  • Status changed from Confirmed to In Progress
  • Assignee set to anonym
  • % Done changed from 0 to 20

While testing this (#7212) I noticed the following:

  • Setup:
    - Unsurprisingly, all packages can be installed straight from Jessie, so no -t or $pkg/testing is needed, and we can drop the custom apt-pinnings. Yeah! We should make it clear that we need non-free sources since ovmf is there.
    - The libavcodec-extra-53 and ffmpeg packages are not in Jessie, so we need to migrate to libav-tools' avconv.
    - The libvirt-ruby package has been renamed to ruby-libvirt.
  • Running:
    - We must get rid of the TailsBootSplash*PostReset.png workaround (the PostReset images should now become the default ones for non-UEFI boots) since virt-viewer now properly applies fullscreen mode on the first boot, not only after a reset. Good news!
    - Similar to the above point, something isn't quite right with the TailsBootSplash*UEFI.png images, but I suspect they only need a trivial update.
    - In features/support/helpers/display_helper.rb, Display.active? always fails. Just removing the call of it in Display.start makes it all run (but may introduce races, especially when snapshots are restored). I tried running active?'s xprop command manually, and it works, so I'm not sure what the problem is. I'm confident we'll find a solution given more than the three minutes I invested in this investigation.

All-in-all it looks like an easy transition.

Bumping the progress slightly to reflect this research.

#4 Updated by anonym almost 5 years ago

anonym wrote:

  • Setup:

Almost forgot this one:
- Instead of syslinux we must install syslinux-utils since isohybrid (used by untrusted_partitions.feature) has moved there.

#5 Updated by intrigeri almost 5 years ago

wrote (06 Nov 2014 14:06:56 GMT) :

  • Setup:
    - Unsurprisingly, all packages can be installed straight from Jessie, so no -t or
    $pkg/testing is needed, and we can drop the custom apt-pinnings. Yeah! We should
    make it clear that we need non-free sources since ovmf is there.
    - The libavcodec-extra-53 and ffmpeg packages are not in Jessie, so we need to migrate to libav-tools' avconv.
    - The libvirt-ruby package has been renamed to ruby-libvirt.

IIRC I've already adjusted most of that in the feature/jessie branch.

All-in-all it looks like an easy transition.

:)

#6 Updated by intrigeri almost 5 years ago

I think I've now pushed all test suite setup doc adjustments to feature/jessie, except:

The libavcodec-extra-53 and ffmpeg packages are not in Jessie, so we need to migrate to libav-tools' avconv.

I suspect we'll want libavcodec-extra-56 and avconv.

anonym, have you got something to push too, with the adaptations needed for runtime?

#7 Updated by anonym almost 5 years ago

intrigeri wrote:

anonym, have you got something to push too, with the adaptations needed for runtime?

No, I only (without committing) removed those deps from the dep checking, and let the steps depending on the video fail.

#8 Updated by intrigeri over 4 years ago

  • Priority changed from Normal to Elevated

#10 Updated by anonym over 4 years ago

  • Target version changed from Tails_2.0 to Tails_1.5

#11 Updated by anonym over 4 years ago

  • Target version changed from Tails_1.5 to Tails_1.4

#12 Updated by intrigeri over 4 years ago

  • Blocks Feature #5683: Puppet module to deploy systems able to run our test suite added

#13 Updated by anonym over 4 years ago

  • Target version changed from Tails_1.4 to Tails_1.3.2

#14 Updated by anonym over 4 years ago

  • Blocked by Bug #9074: python-xmpp broken in Jessie which breaks otr-bot added

#15 Updated by anonym over 4 years ago

  • % Done changed from 20 to 40
  • Feature Branch set to test/8165-run-test-suite-in-jessie
  • Type of work changed from Code to Wait

In the feature branch I've fixed everything in #8165#note-3 (and some more fixes), and I've seen all features pass completely, except pidgin.feature due to #9074 (and transient issues due to #9072) so what remains is waiting until it's fixed.

#16 Updated by intrigeri over 4 years ago

  • Type of work changed from Wait to Code

(The "blocked by" relationship is already expressed in a better way.)

#17 Updated by anonym over 4 years ago

  • Assignee changed from anonym to intrigeri
  • % Done changed from 40 to 50
  • QA Check set to Info Needed

With #9072 resolved, I think this branch is ready to be merged. However, Debian Jessie hasn't been released yet, so I'm unsure if we want this to go into our stable branches just yet. In any case, just to avoid any problems this late in the Tails 1.3.2 release, I'd prefer to do this transition after the release.

What do you think?

#18 Updated by intrigeri over 4 years ago

With #9072 resolved, I think this branch is ready to be merged.

\o/

However, Debian Jessie hasn't been released yet, so I'm unsure if we want this to go into our stable branches just yet.

I say let's merge it into devel after 1.3.2 is out, which will give everyone affected 4-5 weeks to upgrade before 1.4~rc1 has to be tested. It's also likely that Jessie is released during our 1.4 dev cycle.

#19 Updated by intrigeri over 4 years ago

  • Assignee changed from intrigeri to anonym
  • Target version changed from Tails_1.3.2 to Tails_1.4
  • QA Check deleted (Info Needed)

#20 Updated by anonym over 4 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check set to Ready for QA

#21 Updated by intrigeri over 4 years ago

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed

First problem I've seen: CTRL-C'ing a running test suite doesn't seem to work anymore: the TightVNC display turns black, QEMU is still running, and cucumber is eating a full CPU core. I had to pkill -9 cucumber, and then virsh destroy TailsToaster.

I've now started a full run, will report back later if I find any other regression.

#22 Updated by intrigeri over 4 years ago

Note to myself -- check what happens exactly wrt. the guestfs "appliance":

  • on initial setup on Jessie
  • on upgrade from Wheezy

#23 Updated by intrigeri over 4 years ago

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:

First problem I've seen: CTRL-C'ing a running test suite doesn't seem to work anymore:

A 2nd CTRL-C is actually enough, so no problem.

I've now started a full run, will report back later if I find any other regression.

Completed, I now need to analyze the results and sort false positives out..

#24 Updated by intrigeri over 4 years ago

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100

#25 Updated by intrigeri over 4 years ago

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

Merged into stable, devel, feature/jessie and experimental. Will now complete #5683 and send a heads-up on -dev@ so that anyone affected upgrades their testing environment.

#26 Updated by BitingBird over 4 years ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF