Reconfigure Deployment Process

[mekarpeles]

Proposal

Redesign deployment process to be performed from staff developer machines rather than ol-home0, move the docker build process into a github action, and consider moving olsystem to git lab internal.

Justification

Currently, deploys are broken / halted.

As a result of recent security changes that disable ssh-forwarding on hosts, deploys cannot easily be kicked off from ol-home0 as a central host. This is how our current deployment flow is designed to work.

Instead, we should plan on deploys being kicked off by staff (who have sufficient access to cluster nodes) from their local machines.

This is the minimal path to unblock our deployment process.

We're trying to do things in a similar way to @samuel-archive @traceypooh and @abezella in a world where ssh-forwarding is by-default unavailable on hosts.

Breakdown

Requirements Checklist

• Update runner script to run from local machines, not ol-home0 and relax checks that require script to be run from ol-home0.
• Tear our docker-build process and have this done weekly and/or on demand via a github action; requires using github secrets for dockerhub @cdrini @jimchamp
• Update scripts to use io0 or nexus http proxies where necessary to pull docker images
• Move olsystem to internal git lab instead of github?

Open Question

• move deployment scripts to olsystem? https://github.com/internetarchive/openlibrary/tree/master/scripts/deployment

Related files

• https://github.com/internetarchive/openlibrary/tree/master/scripts/deployment

---

Instructions for Contributors

Please run these commands to ensure your repository is up to date before creating a new branch to work on this issue and each time after pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.

[mekarpeles]

Please update status here and link to PR