Project

General

Profile

Bug #17423

Decide which incremental upgrade paths we want to manually test as part of our release process

Added by intrigeri 3 months ago. Updated 16 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Test suite
Target version:
-
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Discuss
Blueprint:
Starter:
Affected tool:

Description

Before 1696d48cbdfab990b09ad7ac45a2078dc904e7fe we would test all incremental upgrade paths. To take an extreme example, when we'll release Tails 4.20 there will be more than 2 dozens such upgrade paths. I felt this would take too much of our resources to me, so in 1696d48cbdfab990b09ad7ac45a2078dc904e7fe I changed the doc to only test the "current stable installed from scratch → version being prepared" upgrade path (which uses the v2 upgrader, v2 UDFs, and a v2 IUK). But perhaps that's not good enough and we need to find a middle ground.

Another question is whether we want to test "Tails << 4.2 → 4.2 or 4.2.1 → version being prepared", which is a bit different: the first step uses v1 UDFs and IUKs, while the 2nd step switches to v2 UDFs and IUKs.

After the 4.2.1 release, depending on how things go (now that the IUKs and UDFs have been fixed, presumably, and now that all the UDFs are on the test channel), we can discuss what a suitable trade-off could look like.

History

#1 Updated by CyrilBrulebois 3 months ago

Food for thoughts: #17425

#2 Updated by intrigeri 3 months ago

CyrilBrulebois wrote:

Food for thoughts: #17425

Yep, that's definitely something to take into account.

FWIW, here's an update on this front: we now have regression tests in place for #17425, one in the Upgrader codebase itself, the other in the Tails integration test suite (f46d5c504fac878a43de91e74ad9e000c68cb137 and f122cc078ef4421f1f4d94ecf716620993e5a2ea). Had they been added as part of #15281, we would not have hit #17425 at the worst possible time.

#3 Updated by anonym 16 days ago

  • Status changed from Confirmed to Resolved
During the RM meeting on 2020-03-19 we decided:
  • We thinks it's now enough to do the minimal (as documented) test as long as the Upgrader and IUK generation do not change
  • For less simple situations, we try to figure out in advance what can go wrong, with developers + RM + other relevant stakeholders, we test risky cases before merging, and then the RM adds whatever extra manual tests they feel are needed during the release process.

Also available in: Atom PDF