Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move backend for CDN to S3 or similar #35

Open
ewdurbin opened this issue Sep 13, 2018 · 3 comments
Open

Move backend for CDN to S3 or similar #35

ewdurbin opened this issue Sep 13, 2018 · 3 comments

Comments

@ewdurbin
Copy link
Member

Currently bootstrap.pypa.io's backend is a VM, which has some flexibility in managing how bootstrap.pypa.io is "put together", which is susceptible to all the failings of kernel panics and misconfiguration that can lead to outages.

It might be worth considering migrating the generation of the files to be the responsibility of CI and syncing the result to S3. This would likely be more reliable than even spinning up a second host and would certainly be cheaper.

Ref #34

@ewdurbin
Copy link
Member Author

salt states that "build" bootstrap.pypa.io's assets: https://github.com/python/psf-salt/tree/master/salt/pypa/bootstrap

@dstufft
Copy link
Member

dstufft commented Sep 13, 2018

The main "problem" with that, which isn't really a problem problem, just something to solve is currently the setup pulls from repositories, so we don't need to distribute secrets anywhere (other than VCS secrets, which we need anyways). Switching to a push based model would mean that we need to distribute S3 keys to all of the projects that are currently publishing to bootstrap.pypa.io.

Another option might be to only support Github repositories as a source of truth, and use lambda to listen for push hooks from GitHub, and automatically fetch and upload to S3 in response to them?

@chrahunt
Copy link
Member

chrahunt commented Oct 5, 2019

This should be a bit easier now with GitHub Actions, which can take care of the triggering and holding secrets.

An example of uploading to an s3 bucket on push can be found here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants