Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is it possible to add a command to set the size of chunked uploads #24919

Closed
PunkFleet opened this issue Dec 31, 2024 · 4 comments
Closed

Is it possible to add a command to set the size of chunked uploads #24919

PunkFleet opened this issue Dec 31, 2024 · 4 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@PunkFleet
Copy link

Feature request description

When deploying a private registry locally on a NAS, one common usage scenario is to penetrate through the cloudflare tunnel intranet to the local registry ip address.
However, uploading images at this time is subject to cloudflare's >100M file error, and cloudflare they do allow chunked upload settings on the request side.
I'm just wondering if in the future podman will add a similar setting to facilitate local deployment of private registries?

admin@koala ~ % podman push registry.test.com/test/postgres:latest                  
Getting image source signatures
Copying blob sha256:035a390a8e98a1f63466237957754e9052803a1fa0feb6158b7438e2111cfa84
Copying blob sha256:438e1f452bcf76c12b20cebaa4a9749c0f4cdf0790ce14fca4f2489154e07a3b
Copying blob sha256:98f5ec96c40600998aaa37995fb65076cfbc44038e0cde41aab3b51c2d5b587d
Copying blob sha256:a122021c9b827d14c01dc61b0f31637b525d8dc1964fd4217a9e209f33ec83cb
Copying blob sha256:b23fbef38c25f6bcf364ec63b3455e419d5ee2b344bb49c3d6e45e5deca1bce0
Copying blob sha256:e68cd777824886fb353f76cd7add7a8ead63a5e931f53d25a925abb2ab931a9e
Copying blob sha256:ef3ed172bcbee926dfcd028f7d49c5837ca4ccf8a83c4ce8422a7a9451f7dd8e
Copying blob sha256:51651ec9f34a1bd7303c607f2ead8e4b8c90e85dfef729277fa85df62fe6b02a
Copying blob sha256:25152395b088e71ead37fae38aa45d17bcbd22b839c13f79577b36c9d60d5101
Copying blob sha256:060d87cb898ac01d980860943c5ff812d475a0634683178ad93517c1b4be19a0
Copying blob sha256:5455980effd41a215ce849ce242b59d61f156f9069cd5bfd006d0bd715140bed
Copying blob sha256:d8df6eb237ac6112000ad3012c030825de14a43e7174a388e47c25720372ce19
Copying blob sha256:95cde9738dd93dacc2ee76d8a5459ceb700b4f7f1d7159b1864acc153f09aba6
Copying blob sha256:cea7dcf359206a76616436f61d3b237bddc3e76ba47779d101cc1e93807aa6ca
Error: writing blob: uploading layer chunked: StatusCode: 413, "<html>\r\n<head><title>413 Request Entity Too Large<..."

Suggest potential solution

Change the chunk size via a parameter, for example --chunk-size:

podman push --chunk-size=100M example.com/registry/test:latest

Have you considered any alternatives?

I'm not quite sure if it's technically or security-wise possible to implement it this way

Additional context

Add any other context or screenshots about the feature request here.
@PunkFleet PunkFleet added the kind/feature Categorizes issue or PR as related to a new feature. label Dec 31, 2024
@PunkFleet
Copy link
Author

I mean, is it possible to do the chunk_size setting from the client, the scenario is similar to nextcloud:
nextcloud/desktop#4271 (comment)

@Luap99
Copy link
Member

Luap99 commented Jan 6, 2025

I don't think it makes sense to expose such options on a cli level? Which user would actually know what the correct values for a given registry are and then if they really know these values then I think they would rather set this in a config file once and not for every single push to the registry.

But the better question is does the registry behind the proxy even support uploading files in chunks? I have no idea what the registry spec says on these things and how the real world registries behave.
cc @mtrmac

@mtrmac
Copy link
Collaborator

mtrmac commented Jan 6, 2025

Theoretically, support for chunked uploads exists in the protocol, but IIRC Docker never uses it, and historically it outright didn’t work against the primary registry implementation.

Compare various data points in earlier containers/image#1869 , including the somewhat surprising report that Docker was let through, suggesting that this might be a proxy configuration specifically looking for Docker — i.e. something that should be fixed in the proxy without forcing every user to add an option.

@PunkFleet
Copy link
Author

suggesting that this might be a proxy configuration specifically looking for Docker — i.e. something that should be fixed in the proxy without forcing every user to add an option.

I've been looking for options, and you're right, this really shouldn't be left up to the registry, it adds unnecessary burden to the registry. I figured this out and I will close this request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mtrmac @Luap99 @PunkFleet