Project

General

Profile

Bug #17203

Automatic install of additional software fails because Tails has not yet connected to Tor

Added by cbrownstein about 1 month ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Persistence
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
bugfix/17203-upgrade-additional-software-after-tor-has-bootstrapped
Type of work:
Code
Blueprint:
Starter:
Affected tool:
Additional Software Packages

Description

Sometimes Tails (v4.0) fails to automatically install the additional software I have selected for automatic installation.

I believe the install fails because Tails has not yet connected to Tor when the additional software is attempted to be installed. (I am displayed the "Tor is ready" notification only after the install fails.)

N.B. I encounter no problems installing the additional software with apt after Tails has connected to Tor.

Here is the log I am given after the install fails:

[INFO] Starting to install additional software...
[INFO] Found additional packages list.
[WARNING] Found a copy of old APT lists, restoring it.
[INFO] Will install the following packages: fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager
[INFO] Reading package lists...
[INFO] Building dependency tree...
[INFO] Reading state information...
[INFO] The following package was automatically installed and is no longer required:
[INFO]   libpcre2-posix0
[INFO] Use 'apt autoremove' to remove it.
[INFO] The following additional packages will be installed:
[INFO]   libhtml-scrubber-perl libhtml-template-perl libimage-magick-perl
[INFO]   libimage-magick-q16-perl libjson-perl libmarkdown2 libosp5 libruby2.5
[INFO]   libsgmls-perl libtcl8.6 libtext-markdown-discount-perl libyaml-tiny-perl
[INFO]   opensp rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet
[INFO]   ruby-power-assert ruby-test-unit ruby-xmlrpc ruby2.5 rubygems-integration
[INFO]   sgml-base vim-runtime
[INFO] Suggested packages:
[INFO]   dvipng graphviz libfile-mimeinfo-perl libhighlight-perl libmagickcore-extra
[INFO]   libmailtools-perl libnet-amazon-s3-perl libnet-inet6glue-perl
[INFO]   libsearch-xapian-perl libsort-naturally-perl libsparkline-php
[INFO]   libtext-csv-perl libtext-multimarkdown-perl libtext-textile-perl
[INFO]   libtext-typography-perl libtext-wikicreole-perl libtext-wikiformat-perl
[INFO]   libxml-feed-perl libxml-writer-perl polygen python3-docutils texlive tidy
[INFO]   viewvc | gitweb | viewcvs xapian-omega libipc-sharedcache-perl
[INFO]   imagemagick-doc sgmlspl sgmls-doc tcl8.6 doc-base sgml-data powerline-doc ri
[INFO]   ruby-dev bundler sgml-base-doc cscope vim-doc
[INFO] Recommended packages:
[INFO]   gcc | c-compiler libauthen-passphrase-perl libc6-dev | libc-dev
[INFO]   libcgi-formbuilder-perl libcgi-session-perl libcrypt-ssleay-perl
[INFO]   libgravatar-url-perl liblwpx-paranoidagent-perl libmail-sendmail-perl
[INFO]   libnet-openid-consumer-perl librpc-xml-perl libterm-readline-gnu-perl
[INFO]   libxml-simple-perl libjson-xs-perl libterm-readkey-perl zip fonts-lato
[INFO] The following NEW packages will be installed:
[INFO]   fonts-powerline ikiwiki libhtml-scrubber-perl libhtml-template-perl
[INFO]   libimage-magick-perl libimage-magick-q16-perl libjson-perl libmarkdown2
[INFO]   libosp5 libruby2.5 libsgmls-perl libtcl8.6 libtext-markdown-discount-perl
[INFO]   libyaml-syck-perl libyaml-tiny-perl opensp perlmagick po4a powerline
[INFO]   python3-powerline rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet
[INFO]   ruby-power-assert ruby-test-unit ruby-xmlrpc ruby2.5 rubygems-integration
[INFO]   sgml-base vim-addon-manager vim-nox vim-runtime
[INFO] 0 upgraded, 34 newly installed, 0 to remove and 20 not upgraded.
[INFO] Need to get 9156 kB/17.6 MB of archives.
[INFO] After this operation, 70.5 MB of additional disk space will be used.
[INFO] Err:1 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 libimage-magick-q16-perl amd64 8:6.9.10.23+dfsg-2.1
[INFO]   Timed out while waiting to read 'first part of response' from proxy socks5h://127.0.0.1:9050 [IP: 127.0.0.1 9050]
[INFO] Get:2 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 po4a all 0.55-1 [1542 kB]
[INFO] Get:3 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 python3-powerline all 2.7-2 [126 kB]
[INFO] Get:4 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 powerline amd64 2.7-2 [51.6 kB]
[INFO] Get:5 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 vim-addon-manager all 0.5.10 [19.9 kB]
[INFO] Get:6 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 vim-runtime all 2:8.1.0875-5 [5775 kB]
[INFO] Get:7 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 vim-nox amd64 2:8.1.0875-5 [1401 kB]
[INFO] Fetched 8866 kB in 1min 14s (120 kB/s)
[INFO] E: Failed to fetch tor+http://vwakviie2ienjx6t.onion/debian/pool/main/i/imagemagick/libimage-magick-q16-perl_6.9.10.23+dfsg-2.1_amd64.deb  Timed out while waiting to read 'first part of response' from proxy socks5h://127.0.0.1:9050 [IP: 127.0.0.1 9050]
[INFO] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
[WARNING] apt-get exited with returncode 100
[WARNING] Warning: installation of fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager failed

Related issues

Related to Tails - Bug #17278: Additional Software tests often fail Confirmed
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

Associated revisions

Revision 3982cb13 (diff)
Added by intrigeri about 1 month ago

Wait until Tor has bootstrapped before we try to upgrade Additional Software (refs: #17203)

This was the intention of this code since the beginning, but unfortunately
we got the unit name wrong, so our After= stances were ineffective.

Revision f5d5e04d
Added by segfault about 1 month ago

Merge branch 'bugfix/17203-upgrade-additional-software-after-tor-has-bootstrapped' into stable (Closes: #17203)

History

#1 Updated by cbrownstein about 1 month ago

I'm not sure if this matters: I upgraded from Tails 3.16 and therefore had old apt lists and old cached packages.

#2 Updated by intrigeri about 1 month ago

Hi Cody,

Sometimes Tails (v4.0) fails to automatically install the additional software I have selected for automatic installation.

Ouch!

I believe the install fails because Tails has not yet connected to Tor when the additional software is attempted to be installed.

The log shows that:

  • the first attempt to download a package fails
  • then the next 6 packages are downloaded successfully

So yeah, it might be that we're starting too early (e.g. the time sync' process may mess with ASP).

To investigate, we would need a WhisperBack report.

#3 Updated by op_mb about 1 month ago

if you're getting this after Tails boots and tries to install aditional software, it means your /var/lib/apt/lists wasn't synchronized on exit (usually if you pull the usb out).

on Tails start, it will look to "download" from your persistent source=apt/lists.
run: sudo apt-update // after you install additional software for the first time.

if that doesnt help, copy and paste manually from /var/lib/apt/lists to source=apt/lists

#4 Updated by op_mb about 1 month ago

correction: sudo apt-get update

#5 Updated by intrigeri about 1 month ago

if you're getting this after Tails boots and tries to install aditional software, it means your /var/lib/apt/lists wasn't synchronized on exit (usually if you pull the usb out).

The log shared by cbrownstein suggests that the APT lists were just fine. But indeed, one relevant question is: why wasn't libimage-magick-q16-perl in the cache already?

I'm not sure if this matters: I upgraded from Tails 3.16 and therefore had old apt lists and old cached packages.

Did this problem happen on first boot on 4.0 after upgrading?
Did this problem happen on any subsequent 4.0 boot?

#6 Updated by cbrownstein about 1 month ago

[...] But indeed, one relevant question is: why wasn't libimage-magick-q16-perl in the cache already?

Wouldn't the version in the cache be from stretch rather than from buster?

Did this problem happen on first boot on 4.0 after upgrading?

Yes, I encountered this problem on my first boot after upgrading from 3.16 to 4.0.

Did this problem happen on any subsequent 4.0 boot?

Yes, I encountered the problem on every subsequent boot until I ran:

sudo apt install fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager

I have not encountered the problem since running that command.

#7 Updated by intrigeri about 1 month ago

  • Status changed from New to Confirmed
  • Assignee set to intrigeri
  • Target version set to Tails_4.1
  • Affected tool set to Additional Software Packages

#8 Updated by intrigeri about 1 month ago

  • Status changed from Confirmed to In Progress
  • Feature Branch set to bugfix/17203-upgrade-additional-software-after-tor-has-bootstrapped

#9 Updated by intrigeri about 1 month ago

Hi,

[...] But indeed, one relevant question is: why wasn't libimage-magick-q16-perl in the cache already?

Wouldn't the version in the cache be from stretch rather than from buster?

OK, I'm now convinced that you're right, thanks to this clarification:

Did this problem happen on first boot on 4.0 after upgrading?

Yes, I encountered this problem on my first boot after upgrading from 3.16 to 4.0.

My current conclusion on this front is:

  • APT could fetch the lists successfully, then Tails did something that disturbed Tor, which caused the download of libimage-magick-q16-perl to fail, then Tor came back up and the other packages could be downloaded without trouble.
  • Regarding the "something that disturbed Tor": to be investigated; potential offenders:
    • ordering of tails-additional-software-upgrade.service vs. Tor bootstrap and time sync'ing
    • the cases where we restart the tor service
    • time sync'ing

A first, obvious bug is that tails-additional-software-upgrade.service has After=tor-has-bootstrapped.service, while we have no such service: it should instead be After=tails-wait-until-tor-has-bootstrapped.service. I don't know if that's the only reason for this bug. I'm tempted to be optimistic, fix this one, and see if more reports about such problems flow in.

Did this problem happen on any subsequent 4.0 boot?

Yes, I encountered the problem on every subsequent boot until I ran:

> sudo apt install fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager
> 

It's a weird coincidence, unless you're always using this Tails on computers and networks with similar performance to each other, in which case, indeed there's a good chance that you hit the exact same race condition every time.

I have not encountered the problem since running that command.

OK. My understanding is that this ensured all the packages are in the cache, so next time, they'll be installed via "Software installation at startup" instead of "Software upgrade on Internet connection" (https://tails.boum.org/contribute/design/additional_software_packages/).

#10 Updated by intrigeri about 1 month ago

#11 Updated by cbrownstein about 1 month ago

Did this problem happen on any subsequent 4.0 boot?

Yes, I encountered the problem on every subsequent boot until I ran:
[...]

It's a weird coincidence, unless you're always using this Tails on computers and networks with similar performance to each other, in which case, indeed there's a good chance that you hit the exact same race condition every time.

FWIW, I was using the exact same computer and network when I was experiencing the problem.

#12 Updated by intrigeri about 1 month ago

  • Status changed from In Progress to Needs Validation
  • Assignee deleted (intrigeri)

#13 Updated by segfault about 1 month ago

  • Status changed from Needs Validation to Resolved
  • % Done changed from 0 to 100

#14 Updated by intrigeri 14 days ago

  • Related to Bug #17278: Additional Software tests often fail added

Also available in: Atom PDF