Project

General

Profile

Feature #10294

Bug #7161: Support more than 24 HTTP mirrors

Define format for per-mirror hostname

Added by intrigeri over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Infrastructure
Target version:
Start date:
09/28/2015
Due date:
% Done:

100%

Feature Branch:
doc/11054-per-mirror-hostname
Type of work:
Research
Blueprint:
Starter:
Affected tool:

Description

For #8635 (at least) we need to decide how to call the mirrors. This also impacts the config file format (#8637) and the script (#8639).

Note that all mirrors will need to answer both on the fallback hostname (likely dl.amnesia.boum.org) and on their own one, so that we can add them to the fast and reliable fallback pool (for non-JS users) without any need for server-side configuration changes.

Also note that whatever per-mirror hostname format we choose must exist, and be maintained, in the DNS.


Related issues

Blocks Tails - Feature #8635: Make each mirror provide a unique virtualhost name Resolved 02/04/2016
Blocks Tails - Feature #8637: Design how to convey the mirror pool's configuration to the dispatcher script Resolved 01/09/2015 04/15/2016
Blocks Tails - Feature #8639: Write a mirror pool dispatcher script Resolved 01/09/2015 04/15/2016

Associated revisions

Revision 731aab8f (diff)
Added by intrigeri over 3 years ago

HTTP mirroring doc: support per-mirror hostnames.

refs: #10294, #11054

History

#2 Updated by intrigeri over 3 years ago

  • Parent task set to #7161

#3 Updated by intrigeri over 3 years ago

  • Blocks Feature #8635: Make each mirror provide a unique virtualhost name added

#4 Updated by intrigeri over 3 years ago

  • Blocks Feature #8637: Design how to convey the mirror pool's configuration to the dispatcher script added

#5 Updated by intrigeri over 3 years ago

  • Blocks Feature #8639: Write a mirror pool dispatcher script added

#6 Updated by intrigeri over 3 years ago

Ideally the format we pick should work for Apache, nginx and lighttpd. If it's hard to support all three then we should check popularity of those webservers among our current pool, or find a technical solution to support multiple formats.

#7 Updated by intrigeri over 3 years ago

  • Assignee changed from u to intrigeri

#8 Updated by u over 3 years ago

current research seems to confirm that the option *.1.dl.amnesia.boum.org is suitable.

nginx

http://nginx.org/en/docs/http/server_names.html

A wildcard name may contain an asterisk only on the name’s start or end, and only on a dot border. The names “www.*.example.org” and “w*.example.org” are invalid. However, these names can be specified using regular expressions, for example, “~^www\..+\.example\.org$” and “~^w.*\.example\.org$”. An asterisk can match several name parts. The name “*.example.org” matches not only www.example.org but www.sub.example.org as well.

A special wildcard name in the form “.example.org” can be used to match both the exact name “example.org” and the wildcard name “*.example.org”.

apache

https://httpd.apache.org/docs/2.4/vhosts/name-based.html#using

The wildcard characters * and ? can be used to match names.

lighthttpd

(unsure about details) http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_Configuration

  # handle virtual hosting
  # map all domains of a top-level-domain to a single document-root
  $HTTP["host"] =~ "(^|\.)example\.org$" {
    server.document-root = "/var/www/htdocs/example.org/pages/" 
  }

#9 Updated by intrigeri over 3 years ago

  • Description updated (diff)
  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 20

u wrote:

nginx

http://nginx.org/en/docs/http/server_names.html

This should work: server_name dl.amnesia.boum.org *.dl.amnesia.boum.org;
Let's now test it.

apache

Seems that something like this should work:

ServerName dl.amnesia.boum.org
ServerAlias *.dl.amnesia.boum.org

Needs to be tested, especially since it seems to only work for name-based virtual hosts, while our current example config snippet is an IP-based virtual host.

lighthttpd

(unsure about details) http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_Configuration

[...]

Our current example config snippet reads:

$HTTP["host"] =~ "^dl\.amnesia\.boum\.org$" {

... so replacing it with something like:

$HTTP["host"] =~ "^(\d+\.)?dl\.amnesia\.boum\.org$" {

... should work.

#10 Updated by sajolida over 3 years ago

Did you consider moving to dl.tails.boum.org and *.dl.tails.boum.org while working on this move?

#11 Updated by intrigeri over 3 years ago

Did you consider moving to dl.tails.boum.org and *.dl.tails.boum.org while working on this move?

Not directly, but last time I thought about it, it seemed useful to keep what we don't control much (mirrors) under a different hierarchy than our website and the infra we control, e.g. for HSTS and friends. Otherwise we may have to tightly couple HTTPS hardening of our services with moving all mirrors to HTTPS. Both are desirable, but tight coupling sounds like the wrong strategy. If there's more discussion to be had on this topic, let's have it on a dedicated ticket, since it's a little bit off-topic here and we're almost done with what this ticket is about :)

#12 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_1.7 to Tails_1.8

#13 Updated by intrigeri over 3 years ago

I'll focus on #5926 instead, during the 1.8 cycle.

#14 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_1.8 to 246

#15 Updated by sajolida over 3 years ago

  • Target version changed from 246 to Tails_2.0

#16 Updated by intrigeri over 3 years ago

  • Target version changed from Tails_2.0 to Tails_2.2

#17 Updated by intrigeri over 3 years ago

  • Description updated (diff)

#18 Updated by intrigeri over 3 years ago

  • Description updated (diff)

#19 Updated by intrigeri over 3 years ago

  • Feature Branch set to doc/11054-per-mirror-hostname

#20 Updated by intrigeri over 3 years ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 20 to 100

Tested all these config snippets, they work fine => the format will be *.dl.amnesia.boum.org. I'll follow-up on #11054.

Also available in: Atom PDF