Project

General

Profile

Bug #15918

Move Redmine out of *.riseup.net

Added by intrigeri 3 months ago. Updated 3 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Infrastructure
Target version:
Start date:
09/06/2018
Due date:
% Done:

100%

Spent time:
QA Check:
Feature Branch:
puppet-tails:feature/15918-remove-old-labs-vhost
Type of work:
Sysadmin
Blueprint:
Starter:
Affected tool:

Description

Our usage of the labs.r.n hostname complicates things for Riseup and breaks some stuff (e.g. sending email from riseup.net to Redmine). We've been asked to stop using that hostname soon. We can have redirects to avoid breaking existing URLs (most likely the rewrite rules will live in Apache).

Migration plan

stop using labs.riseup.net

  1. choose a new X.tails.boum.org FQDN: redmine.tails.boum.org
  2. get DNS set up for the new FQDN
  3. add support for the new FQDN in our webserver config
  4. get a LE cert for the new FQDN
  5. switch Redmine config to the new FQDN
  6. update all URLs and hostnames we control to point the new FQDN (website, monitoring, Puppetized stuff)
  7. fix https://redmine.tails.boum.org/ homepage
  8. adjust /etc and other non-Puppetized places where labs.riseup.net might be hard-coded
  9. milestone: the official URL for our Redmine uses the new FQDN
  10. make outgoing email be sent From: redmine@redmine.tails.boum.org
  11. have Riseup folks ensure email sent to will keep being handled in a useful manner: ideally, being keep sent to buse; worst case, helpful SMTP error that hints the sender
  12. have Riseup folks set up a webserver that answers requests to https://labs.riseup.net/(.*) with a redirection to https://redmine.tails.boum.org/$1
  13. have Riseup folks point the DNS for labs.riseup.net to their redirector
  14. have Riseup folks update the reverse DNS for our IP to point to the new FQDN
  15. milestone: we don't own labs.riseup.net anymore
  16. drop support for labs.riseup.net in our webserver config

stop using buse.riseup.net: rename the machine to buse.tails.boum.org

  1. have DNS set up for buse.tails.b.o
  2. adjust Puppet, deploy
  3. adjust /etc and other non-Puppetized places where "buse" might be hard-coded (postfix, munin)
  4. have Riseup folks delete the DNS record for buse.riseup.net

Related issues

Blocks Tails - Feature #13284: Core work 2017Q2→2019Q2: Sysadmin (Adapt our infrastructure) Confirmed 06/30/2017

Associated revisions

Revision 276f0be4 (diff)
Added by intrigeri about 1 month ago

Website: migrate URLs from labs.riseup.net to redmine.tails.boum.org (refs: #15918)

Here I'm intentionally skipping:

- Anything that's outside of wiki/src/, e.g. code and test suite
- Historical content: blog posts, meeting notes and logs
- Wiki-style content: blueprints

History

#1 Updated by intrigeri 3 months ago

  • Blocks Feature #13284: Core work 2017Q2→2019Q2: Sysadmin (Adapt our infrastructure) added

#2 Updated by intrigeri 3 months ago

  • Category set to Infrastructure

#3 Updated by intrigeri 3 months ago

  • Description updated (diff)

#4 Updated by intrigeri 3 months ago

  • Target version changed from Tails_3.12 to Tails_3.11

I'll work on this during our probable sysadmin sprint/meeting in December.

#5 Updated by intrigeri 3 months ago

  • Description updated (diff)

#6 Updated by intrigeri about 1 month ago

  • Description updated (diff)
  • Status changed from Confirmed to In Progress

#7 Updated by intrigeri about 1 month ago

groente, could you please review the migration plan? Meanwhile, I'll get started with the first, less risky steps.

#8 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#9 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#10 Updated by groente about 1 month ago

First comment: i would strongly suggest creating a separate vhost in step 3 instead of using serveralias. Two reasons: 1, there's a lot of redirection and crazyness to old riseup stuff that's not needed for our new name - a new vhost will clean this up. 2, a separate vhost comes with a separate ssl key&cert, so no need to hand over t.b.o. keys and certs to riseup in step 10.

#11 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#12 Updated by intrigeri about 1 month ago

test!

#13 Updated by intrigeri about 1 month ago

  • Description updated (diff)
  • % Done changed from 0 to 10

#14 Updated by groente about 1 month ago

  • % Done changed from 10 to 0

Regarding #5, I think getting rid of the /code baseurl is going to be more of a hassle than it's worth.

You'll end up having to do rewrites not only for labs.riseup.net, but also for redmine.t.b.o, as all the current links in redmine are relative. All the current tickets will end up with links that will send the browser to https://redmine.tails.boum.org/code/whatever , so they will also need to be rewritten.

Imho it'd be less risky and painful to just remove the old stuff in /var/www/ (everything that is not /code) and place a new maintenance.html and index.html with a redirect there.

#15 Updated by intrigeri about 1 month ago

I think getting rid of the /code baseurl is going to be more of a hassle than it's worth.

Agreed.

#16 Updated by intrigeri about 1 month ago

  • Description updated (diff)
  • % Done changed from 0 to 10

#17 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#18 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#19 Updated by intrigeri about 1 month ago

Test sending email to .

#20 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#21 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#22 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#23 Updated by intrigeri about 1 month ago

  • Description updated (diff)

#24 Updated by intrigeri about 1 month ago

  • Description updated (diff)
  • % Done changed from 10 to 50

The official URL for our Redmine uses the new FQDN, the old one redirects to the new one, and we don't use {labs,buse}.riseup.net internally anymore. All that's left is communicating/coordinating with Riseup and the deal with the fallout (e.g. the test suite might be broken).

#25 Updated by intrigeri about 1 month ago

  • Description updated (diff)
  • Assignee changed from intrigeri to groente
  • Priority changed from Elevated to Normal
  • QA Check set to Ready for QA

I've completed the work Riseup asked us to do and asked Riseup to do their bits whenever they see fit => there's no justification for priority > normal anymore.

groente, I think it's a good time for you to review my work. It's all tracked in our manifests repo (+ the corresponding submodule updates) and in buse's etckeeper. All these changes happened between Nov 6 and Nov 9, inclusive.

#26 Updated by groente about 1 month ago

  • Assignee changed from groente to intrigeri
  • % Done changed from 50 to 60
  • QA Check changed from Ready for QA to Pass

that all looks sensible and seems to work :)

#27 Updated by intrigeri about 1 month ago

  • QA Check deleted (Pass)

Thanks!

#28 Updated by intrigeri 22 days ago

  • Description updated (diff)

#29 Updated by intrigeri 22 days ago

test

#30 Updated by intrigeri 22 days ago

  • Description updated (diff)

#31 Updated by intrigeri 6 days ago

  • Description updated (diff)
  • % Done changed from 60 to 70

All done, except the redirector run by Riseup is broken today.

#32 Updated by intrigeri 6 days ago

  • Description updated (diff)
  • Assignee changed from intrigeri to groente
  • % Done changed from 70 to 80
  • QA Check set to Ready for QA
  • Feature Branch set to puppet-tails:feature/15918-remove-old-labs-vhost

The redirector was fixed. I've implemented the only remaining bit in a topic branch, please review and deploy if happy.

#33 Updated by groente 6 days ago

i've merged the branch into puppet-tails, but haven't deployed yet, will do that lateron.

eitherway, i think some more work is needed before we can close this ticket:

- resolv.conf still thinks its living in riseup land
- postfix still relies on the labs.riseup.net ssl certificate
- postfix still has labs.riseup.net in mydestinations
- labs.riseup.net is still in the letsencrypt renewal list

i'll leave it up to you whether you want to do this in puppet or by hand...

#34 Updated by groente 6 days ago

  • Assignee changed from groente to intrigeri
  • QA Check changed from Ready for QA to Dev Needed

#35 Updated by intrigeri 5 days ago

i've merged the branch into puppet-tails, but haven't deployed yet, will do that lateron.

FTR you've deployed this yesterday.

eitherway, i think some more work is needed before we can close this ticket:

Thanks! I indeed forgot to go through this one last time.

- postfix still relies on the labs.riseup.net ssl certificate
- postfix still has labs.riseup.net in mydestinations

The MX for labs.riseup.net points to this machine so at the very least, it needs to keep accepting email sent to this domain for backwards compatibility.

i'll leave it up to you whether you want to do this in puppet or by hand...

I'll do it with Puppet at least for things we manage with Puppet on that box already; and for everything else, I'll see if it's worth puppetizing while I'm at it.

#36 Updated by intrigeri 5 days ago

  • Assignee changed from intrigeri to groente
  • QA Check changed from Dev Needed to Info Needed

- resolv.conf still thinks its living in riseup land

We have (almost) the same domain and search directives on lizard which makes this slightly out of scope for this ticket, which is why I did not bother initially. But let's clean this up while we're at it! I think they are leftovers from the initial Debian installation, many years ago. I don't see what can possibly be relying on these so I propose we drop them on both machines at once. I would do it with Puppet, ensuring these 2 files contain only one line: the nameserver we need. What do you think?

- postfix still relies on the labs.riseup.net ssl certificate

… which expired 2 years ago, and on lizard we use the default settings (self-signed snakeoil certificate), so same reasoning as resolv.conf: arguably out of scope here but yeah, let's clean this up! Done with commit df5d2f72737c4adf2470b7343d690e03a3deaa91 in our manifests repo, deployed. If this comment sent over email lands on Redmine and I get a notification about it, it means that change didn't break the primary use case for this Postfix.

- postfix still has labs.riseup.net in mydestinations

I think this is correct, as per my previous comment.

- labs.riseup.net is still in the letsencrypt renewal list

Done directly on buse with commit 9c09f47e (iirc this file was never managed by Puppet before).

#37 Updated by intrigeri 5 days ago

- postfix still relies on the labs.riseup.net ssl certificate

… which expired 2 years ago, and on lizard we use the default settings (self-signed snakeoil certificate), so same reasoning as resolv.conf: arguably out of scope here but yeah, let's clean this up! Done with commit df5d2f72737c4adf2470b7343d690e03a3deaa91 in our manifests repo, deployed. If this comment sent over email lands on Redmine and I get a notification about it, it means that change didn't break the primary use case for this Postfix.

Works fine :)

#38 Updated by groente 5 days ago

  • Assignee changed from groente to intrigeri
  • QA Check changed from Info Needed to Dev Needed

intrigeri wrote:

- resolv.conf still thinks its living in riseup land

We have (almost) the same domain and search directives on lizard which makes this slightly out of scope for this ticket, which is why I did not bother initially. But let's clean this up while we're at it! I think they are leftovers from the initial Debian installation, many years ago. I don't see what can possibly be relying on these so I propose we drop them on both machines at once. I would do it with Puppet, ensuring these 2 files contain only one line: the nameserver we need. What do you think?

sounds good!

- postfix still relies on the labs.riseup.net ssl certificate

… which expired 2 years ago, and on lizard we use the default settings (self-signed snakeoil certificate), so same reasoning as resolv.conf: arguably out of scope here but yeah, let's clean this up! Done with commit df5d2f72737c4adf2470b7343d690e03a3deaa91 in our manifests repo, deployed. If this comment sent over email lands on Redmine and I get a notification about it, it means that change didn't break the primary use case for this Postfix.

ouch! thanks for the cleanup :)

- postfix still has labs.riseup.net in mydestinations

I think this is correct, as per my previous comment.

indeed it is.

- labs.riseup.net is still in the letsencrypt renewal list

Done directly on buse with commit 9c09f47e (iirc this file was never managed by Puppet before).

great!

#39 Updated by intrigeri 5 days ago

  • Assignee changed from intrigeri to groente
  • QA Check changed from Dev Needed to Ready for QA

- resolv.conf still thinks its living in riseup land

We have (almost) the same domain and search directives on lizard which makes this slightly out of scope for this ticket, which is why I did not bother initially. But let's clean this up while we're at it! I think they are leftovers from the initial Debian installation, many years ago. I don't see what can possibly be relying on these so I propose we drop them on both machines at once. I would do it with Puppet, ensuring these 2 files contain only one line: the nameserver we need. What do you think?

sounds good!

Done with 6ca2602 in our manifests repo. I've pondered using a new Puppet module to do this (and picked https://forge.puppet.com/stm/resolv_conf) before judging the task at hand was not worth the additional complexity and maintenance overhead; same for moving this code to a custom class configured with Hiera; so I ended up doing it with a mere file resource. If you disagree, I'm fine with adding abstraction layers.

#40 Updated by groente 3 days ago

  • Status changed from In Progress to Resolved
  • Assignee deleted (groente)
  • % Done changed from 80 to 100
  • QA Check deleted (Ready for QA)

Also available in: Atom PDF