Project

General

Profile

Feature #15211

Reduce our Chutney network

Added by anonym almost 2 years ago. Updated over 1 year ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
-
Start date:
01/22/2018
Due date:
% Done:

0%

Feature Branch:
test/15211-minimal-tor-network+force-all-tests
Type of work:
Code
Blueprint:
Starter:
Affected tool:

Description

Rationale: the network size is proportional to the RAM/CPU usage, so we shouldn't run a network larger than we need. (Note: the Tor 0.3.2 series that just went stable seems to increase relay memory usage quite drastically for us, so this what was prompted the interest in this ticket.)

Currently we have:
NODES = Authority.getN(4) + \
        BridgeAuthority.getN(1) + \
        NonExitRelay.getN(20) + \
        ExitRelay.getN(10) + \
        Bridge.getN(3) + \
        BridgeObfs4.getN(3) + \
        OnionService.getN(1) + \
        Client.getN(1)

but we should be able to get away with the smallest possible network:
NODES = Authority.getN(1) + \
        BridgeAuthority.getN(1) + \
        NonExitRelay.getN(2) + \
        ExitRelay.getN(1) + \
        Bridge.getN(1) + \
        BridgeObfs4.getN(1) + \
        OnionService.getN(1) + \
        Client.getN(1)

or similar. Some thoughts:
  • What is the lowest number of authorities (incl. bridge authorities) needed for a consensus?
  • 2 non-exits + 1 exit is enough, but maybe we want some diversity? I actually cannot come up with any technical argument for this.
  • I guess testing one bridge at a time is enough -- at least in the current code (ours, Tor Launcher) there's no difference between the 1 vs many cases.

Related issues

Blocked by Tails - Bug #16792: Upgrade our Chutney fork Needs Validation

Associated revisions

Revision de840603 (diff)
Added by anonym almost 2 years ago

Reduce the Chutney Tor network to its minimal size.

The old values were picked pretty much arbitrarily, but with some
effort to make it seem realistic by enabling e.g. some circuit
variety. If we want that, we should specify better the aspects of the
real network that we want to keep. Until then, let's just make it as
small as possible for performance reasons; lately (especially since
tor 0.3.2.x) we've seen increased CPU and memory usage by each tor
process, to the point where it is noticeable on some systems we run it
on. Let's try to be nicer to them!

Refs: #15211

History

#1 Updated by anonym almost 2 years ago

#2 Updated by anonym almost 2 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 20
  • Feature Branch set to test/15211-minimal-tor-network+force-all-tests

I suspect this is the minimal network:

NODES = Authority.getN(1) + \
        NonExitRelay.getN(2) + \
        ExitRelay.getN(1) + \
        Bridge.getN(1) + \
        BridgeObfs4.getN(1) + \
        OnionService.getN(1) + \
        Client.getN(1)

bit I haven't tried a full run yet. I've pushed a branch that will run it on Jenkins.

#3 Updated by anonym almost 2 years ago

Full runs on Jenkins:

I only did a quick analysis, but it seems like all failures are unrelated (<= none occur in both runs).

It is still conceivable that this branch reduces the reliability of the Tor network, and that we haven't seen enough runs to expose that. But my hope is that what we will see is a reduction of failures that we suspect are due to too resource starvation (CPU, memory) on the isotesters.

#4 Updated by anonym almost 2 years ago

  • % Done changed from 20 to 0

anonym wrote:

It is still conceivable that this branch reduces the reliability of the Tor network, and that we haven't seen enough runs to expose that. But my hope is that what we will see is a reduction of failures that we suspect are due to too resource starvation (CPU, memory) on the isotesters.

I started four more full test suite runs. When done we'll have six runs in total, that I'll analyze in detail w.r.t. this.

#5 Updated by anonym over 1 year ago

Still looks bad. I'll bump the number of authorities from 1 to 3 just to see what happens.

#6 Updated by anonym over 1 year ago

  • Target version changed from Tails_3.6 to Tails_3.7

anonym wrote:

Still looks bad. I'll bump the number of authorities from 1 to 3 just to see what happens.

Might have improved it a bit, but seemingly not satisfactory.

#7 Updated by intrigeri over 1 year ago

anonym wrote:

Note: the Tor 0.3.2 series that just went stable seems to increase relay memory usage quite drastically for us

Are these problems tracked somewhere upstream? Perhaps they would be good candidates the tails-needs keyword.

#8 Updated by intrigeri over 1 year ago

  • Target version changed from Tails_3.7 to Tails_3.8

#9 Updated by intrigeri over 1 year ago

  • Target version deleted (Tails_3.8)

#10 Updated by intrigeri over 1 year ago

Note that our chutney submodule is currently 125 commits (=~ 3 years) behind upstream's. I see some more preconfigured minimal network flavours were added. Maybe a first next step would be to update our submodule.

#11 Updated by intrigeri over 1 year ago

#12 Updated by intrigeri 3 months ago

  • Blocked by Bug #16792: Upgrade our Chutney fork added

Also available in: Atom PDF