Project

General

Profile

Feature #17296

Stop testing DVDs manually

Added by sajolida about 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:

100%

Feature Branch:
Type of work:
Contributors documentation
Blueprint:
Starter:
Affected tool:

Description

We have in our manual test suite:

DVD

    Boot on bare-metal from DVD, measuring the boot time (from the syslinux menu until the GNOME desktop is ready — quickly press ENTER in the Greeter).
    Compare with the boot time of the previous Tails version. The new one should not be significantly slower to start.

kibi reported that RMs don't test it on bare metal anymore (maybe the description should be adjusted) but still burn a DVD and run it through a VM. It still takes around 15 minutes each time according to him.

DVDs are used by less than 5% of our users these days. The 2 data points that we have about this are:

  • Click on the download pages. Putting aside /install/download, /install/download-iso, and upgrades, these are the number of clicks on the download buttons:
    Image    Torrent        
Win     24482   6211    30693   67%
Linux    6374   1172     7546   16%
macOS    2922    490     3412    7%
VM       2224    547     2771    6%
DVD      1222    257     1479    3%
        37224   8677    45901    

(Off-topic: Windows users are downloading much more than that are using Tails according to https://tails.boum.org/blueprint/veracrypt/, 67% vs 52%, and we probably have a retention problem here. Or maybe Linux users are more old timers and don't go through the installation scenario so much anymore compared to Windows users).

  • WhisperBack reports in 2019Q1-3:
USB     472  92%
DVD      21   4%
VM       19   4%
        512

Another question is whether or not this small minority is important enough for us to keep doing these manual tests. My guess is that it is not, for example Secure Drop people are using USB sticks, see #17178. Also, none of our personas would prefer DVDs over USB sticks because they all need Persistence (Kim maybe a bit less than Riou and Cris).

Associated revisions

Revision ef2f8911 (diff)
Added by intrigeri about 1 month ago

Stop testing boot time from DVD manually (Closes: #17296)

History

#1 Updated by sajolida about 2 months ago

  • Description updated (diff)

#2 Updated by intrigeri about 1 month ago

Another questions is whether or not this small minority is important enough for us to keep doing these manual tests. My guess is that it is not, […]

I'm fine with dropping this manual test but I'm wondering if I misunderstood something: you wrote "Another questions", and I don't see what's the other question this "another" is refering to.

#3 Updated by sajolida about 1 month ago

  • Description updated (diff)

Fixed "Another questions" → "Another question".

Having only a small minority rely on something shouldn't be an argument in itself to deprioritize it. For example, even a small minority could be really important to us. That's why I didn't want to only mention the numbers and leave this (single) question open for more comments. I personally don't think that this DVD minority is really important to us.

#4 Updated by intrigeri about 1 month ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF