Stop testing DVDs manually
We have in our manual test suite:
DVD Boot on bare-metal from DVD, measuring the boot time (from the syslinux menu until the GNOME desktop is ready — quickly press ENTER in the Greeter). Compare with the boot time of the previous Tails version. The new one should not be significantly slower to start.
kibi reported that RMs don't test it on bare metal anymore (maybe the description should be adjusted) but still burn a DVD and run it through a VM. It still takes around 15 minutes each time according to him.
DVDs are used by less than 5% of our users these days. The 2 data points that we have about this are:
- Click on the download pages. Putting aside /install/download, /install/download-iso, and upgrades, these are the number of clicks on the download buttons:
Image Torrent Win 24482 6211 30693 67% Linux 6374 1172 7546 16% macOS 2922 490 3412 7% VM 2224 547 2771 6% DVD 1222 257 1479 3% 37224 8677 45901
(Off-topic: Windows users are downloading much more than that are using Tails according to https://tails.boum.org/blueprint/veracrypt/, 67% vs 52%, and we probably have a retention problem here. Or maybe Linux users are more old timers and don't go through the installation scenario so much anymore compared to Windows users).
- WhisperBack reports in 2019Q1-3:
USB 472 92% DVD 21 4% VM 19 4% 512
Another question is whether or not this small minority is important enough for us to keep doing these manual tests. My guess is that it is not, for example Secure Drop people are using USB sticks, see #17178. Also, none of our personas would prefer DVDs over USB sticks because they all need Persistence (Kim maybe a bit less than Riou and Cris).
#2 Updated by intrigeri about 1 month ago
Another questions is whether or not this small minority is important enough for us to keep doing these manual tests. My guess is that it is not, […]
I'm fine with dropping this manual test but I'm wondering if I misunderstood something: you wrote "Another questions", and I don't see what's the other question this "another" is refering to.
#3 Updated by sajolida about 1 month ago
- Description updated (diff)
Fixed "Another questions" → "Another question".
Having only a small minority rely on something shouldn't be an argument in itself to deprioritize it. For example, even a small minority could be really important to us. That's why I didn't want to only mention the numbers and leave this (single) question open for more comments. I personally don't think that this DVD minority is really important to us.