Migrate FR prod on new server

[RachL]

Description

New VPS is here : https://www.ovh.com/manager/#/dedicated/vps/vps-d2442729.vps.ovh.net/dashboard

Migration is scheduled for Monday 19th 2AM CET.

Slack channel fr-upgrade

Acceptance Criteria & Tests

1. FR prod is running on at least Ubuntu 20
2. FR prod is running on a server with swap and 16 giga RAM

[mkllnk]

Documenting my steps. First we need to redirect certificate authentications to the new server:

ansible-playbook --vault-password-file ~/.ansible-vault-password --extra-vars "@../ofn-secrets/fr-prod/secrets.yml" -l fr-prod playbooks/letsencrypt_proxy.yml -e "proxy_target=51.38.38.32"

Then we can change the hosts config file to the new IP address. I also added a swap file. Then provision the new server:

ansible-playbook --vault-password-file ~/.ansible-vault-password --extra-vars "@../ofn-secrets/fr-prod/secrets.yml" -l fr-prod site.yml

[mkllnk]

The first time you run the provisioning there's always an issue with brotli and nginx. I did two things to resolve this:

1. I installed libgd-dev to compile brotli.
2. I ran the nginx install first (with -t nginx) which fails but then brotli can be installed successfully and after that this works.

This happens with every new server but I never took the time to find an automated solution.

[mkllnk]

I also had to install db integrations separately:

ansible-playbook --vault-password-file ~/.ansible-vault-password --extra-vars "@../ofn-secrets/fr-prod/secrets.yml" -l fr-prod playbooks/db_integrations.yml

[mkllnk]

I used steps described in #775 (comment) to activate maintenance mode and copy the database. I encountered #819 but could work around it. Another important detail: clear the cache after the database import.

# ./bin/rails console
Rails.cache.clear