Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bypass codalab server for GCS blob storage uploads #4074

Closed
wwwjn opened this issue Apr 23, 2022 · 2 comments · Fixed by #4119
Closed

bypass codalab server for GCS blob storage uploads #4074

wwwjn opened this issue Apr 23, 2022 · 2 comments · Fixed by #4119
Assignees
@wwwjn
Labels
p1 Do it in the next two weeks.
Milestone
Multiple Bundle S...

Comments

@wwwjn
Copy link
Contributor

wwwjn commented Apr 23, 2022

Similar issue for bypass server download #3974.

Need to add support for bypass server when upload to Azure or GCS.

Azure blob storage manager design: https://docs.google.com/document/d/1l4kOqi9irBjOApmn4E6vlzsjAXDJbetIyVw8gMRHrpU/edit#heading=h.vat4mpk6gcza
@wwwjn wwwjn changed the title Using SAS url/Signed url to bypass server for upload to Azure / GCS Using SAS token/Signed url to bypass server for upload to Azure / GCS Apr 23, 2022
@wwwjn wwwjn self-assigned this Apr 26, 2022
@pranavjain pranavjain added this to the Multiple Bundle Storage milestone Apr 26, 2022
@pranavjain pranavjain added the p1 Do it in the next two weeks. label Apr 26, 2022
@wwwjn wwwjn changed the title Using SAS token/Signed url to bypass server for upload to Azure / GCS Using SAS token/Signed url to bypass server for upload to GCS Apr 26, 2022
@wwwjn wwwjn changed the title Using SAS token/Signed url to bypass server for upload to GCS Using Signed url to bypass server for upload to GCS Apr 26, 2022
@wwwjn wwwjn changed the title Using Signed url to bypass server for upload to GCS bypass codalab server for GCS blob storage uploads Apr 26, 2022
@wwwjn wwwjn mentioned this issue May 31, 2022
Merged
4 tasks
@wwwjn wwwjn linked a pull request Jun 29, 2022 that will close this issue
Bypass server upload for GCS #4119
Merged
4 tasks
@pranavjain
Copy link
Contributor

Blocked by #4201

@wwwjn wwwjn moved this to In Review in Fall'22 Goals Oct 3, 2022
@wwwjn wwwjn moved this from In Review to Blocked in Fall'22 Goals Oct 3, 2022
@wwwjn wwwjn moved this from Blocked to In Review in Fall'22 Goals Oct 3, 2022
@wwwjn
Copy link
Contributor Author

wwwjn commented Oct 3, 2022

09/30 Group meeting update: First test this branch on dev environment, benchmarking upload to GCS, then get it merged to master

Repository owner moved this from In Review to Done in Fall'22 Goals Oct 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wwwjn wwwjn

Labels
p1 Do it in the next two weeks.
Projects
Fall'22 Goals
Status: Done
Milestone
Multiple Bundle Storage
Development

Successfully merging a pull request may close this issue.

Bypass server upload for GCS codalab/codalab-worksheets
2 participants
@pranavjain @wwwjn