Project

General

Profile

Feature #12237

Feature #5688: Tails Server: Self-hosted services behind Tails-powered onion services

Feature #12230: Release Tails Server Beta

Reduce apt update time during first start of Tails Server

Added by segfault about 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
02/14/2017
Due date:
% Done:

100%

Feature Branch:
Type of work:
Research
Blueprint:
Starter:
Affected tool:
Server

Description

Currently this takes between 2 and 10 minutes, depending on the Tor circuit.


Related issues

Related to Tails - Feature #11539: Don't ship the testing and unstable APT sources Rejected 06/20/2016
Related to Tails - Feature #12238: Ship full APT lists in the live file system Confirmed 02/14/2017

History

#2 Updated by segfault about 3 years ago

#3 Updated by segfault about 3 years ago

  • Related to Feature #11539: Don't ship the testing and unstable APT sources added

#4 Updated by segfault about 3 years ago

Ideas:

  • More?

#5 Updated by segfault about 3 years ago

  • Related to Feature #5688: Tails Server: Self-hosted services behind Tails-powered onion services added

#6 Updated by intrigeri about 3 years ago

It should be much faster on Stretch, thanks to optimizations in APT.
I say let's benchmark before we try & solve this :)

#7 Updated by segfault about 3 years ago

It should be much faster on Stretch, thanks to optimizations in APT.

That would be great :)

I say let's benchmark before we try & solve this :)

Sure.

#8 Updated by anonym about 3 years ago

segfault wrote:

Ideas:

  • See the commit 440b9cc73a Completely disable APT translations. from the feature/12238-ship-apt-lists Git branch (of #12238). On Jessie this makes quite a difference.

#9 Updated by anonym about 3 years ago

  • Related to Feature #12238: Ship full APT lists in the live file system added

#10 Updated by anonym about 3 years ago

BTW, I have observed a ~50% slowdown since we switched to the APT onion services.

#11 Updated by anonym about 3 years ago

We also have to take into account that feature/stretch has one Debian version less among the APT sources, namely stable, since it is based on testing. When feature/stretch is released, it will have the same number of Debian versions in its APT sources.

#12 Updated by anonym about 3 years ago

After completely disabling APT translations I benchmarked only the apt update time (running it 20 times) like this:

. /usr/local/lib/tails-shell-library/tor.sh && \
(\
  for x in $(seq 1 20); do \
    rm -fr /var/lib/apt/lists/*; \
    systemctl restart tor@default; \
    until tor_is_working; do sleep 1; done; \
    time apt update; done \
) &> apt.log

and got:

  • Tails 2.10 median time: 1m 15s (for 23.6 MB of data)
  • Tails 3.0~beta1 median time: 50s (for 15.2 MB of data)

Taking my previous comment (#12237#note-11) into account (one less APT source in 3.0~beta1) these results seems consistent with there being no noticeable improvement for apt update, so I guess the optimizations talked about in #12237#note-6 are only about the generation of /var/cache/apt/{src,}pkgcache.bin. I guess it'd be interesting to benchmark the time needed to generate those on stable vs stretch, with and without persistent APT lists, but I still suspect that the slowness is dominated by the fetches over Tor.

I did have several outliers at >= five minutes thanks to slow circuits, and previously I've seen even worse (> 10 minutes). We'll never be able to avoid this unless we ship the lists, so if we don't we absolutely must have better feedback about what is going on with apt update in Tails Server. But even apt itself doesn't have very good feedback (beyond "stuff is still being written to the screen, various numbers are increasing"), and neither does Synaptic, so I'm not sure we can easily achieve that.

#13 Updated by intrigeri about 3 years ago

Thanks for this data!

Taking my previous comment (#12237#note-11) into account (one less APT source in 3.0~beta1) these results seems consistent with there being no noticeable improvement for apt update, so I guess the optimizations talked about in #12237#note-6 are only about the generation of /var/cache/apt/{src,}pkgcache.bin.

This matches what I (vaguely) remember.

I guess it'd be interesting to benchmark the time needed to generate those on stable vs stretch, with and without persistent APT lists, but I still suspect that the slowness is dominated by the fetches over Tor.

IIRC the results sajolida gave me a while ago (using real storage hardware) were in the 30-60 seconds ballpark to for the binary cache generation, so the "dominated by" guess isn't obvious me, and I'd like to see it actually measured. I can do it if you prefer, e.g. if you feel that I'm being unnecessarily painful while insisting for benchmarking the entire operation, as perceived by the user, instead of apt update only.

#14 Updated by segfault about 3 years ago

See the commit 440b9cc73a Completely disable APT translations. from the feature/12238-ship-apt-lists Git branch (of #12238). On Jessie this makes quite a difference.

Awesome, thanks a lot! :)

#15 Updated by sajolida almost 3 years ago

IIRC the results sajolida gave me a while ago (using real storage
hardware) were in the 30-60 seconds ballpark to for the binary cache
generation, so the "dominated by" guess isn't obvious me, and I'd
like to see it actually measured. I can do it if you prefer, e.g. if
you feel that I'm being unnecessarily painful while insisting for
benchmarking the entire operation, as perceived by the user, instead
of apt update only.

Yes, in my experience of additional software, even without downloading
anything, the binary cache generation is extremelly slow and long and
storing the download still make a very bad experience. We want to solve
this in #9059 for the Additional software feature.

But I'm glad to see that Tails Server is affected by this problem as well :)

#16 Updated by segfault almost 3 years ago

  • Affected tool set to Server

#17 Updated by segfault almost 3 years ago

#18 Updated by segfault almost 3 years ago

  • Status changed from Confirmed to Resolved
  • % Done changed from 0 to 100
  • Parent task set to #12230

I merged feature/12238-ship-apt-lists from #12238 and now installing services works without updating the apt lists :)

Also available in: Atom PDF