Add new commands to support uploading and downloading images #1366

qweeah · 2024-04-23T06:30:47Z

What is the version of your ORAS CLI

oras v1.1.0

What would you like to be added?

Had an offline discussion with @shizhMSFT and he suggests to add two commands:

oras backup (alias: snapshot, pull-image) to support saving an image from remote registry to an OCI image layout tar archive.

Usage:
  oras backup --output <oci_layout_path> <reg>/<repo>[{:<tag>|@digest]}] [...]

oras restore (alias: push-image) to support pushing an image from an OCI image layout tar archive to remote registry.

Usage:
  oras restore --output <reg>/<repo> <oci_layout_path>[{:<tag>|@digest]}]

Why is this needed for ORAS?

oras cp is not intuitive to use, e.g. user might use oras push to push images from an OCI layout and succeed with false positive (see #1353). Adding those two wrapper commands of oras cp can help avoid that.

Are you willing to submit PRs to contribute to this feature?

Yes, I am willing to implement it.

The text was updated successfully, but these errors were encountered:

TerryHowe · 2024-04-23T16:16:46Z

Why not backup and restore?

…

On Tue, Apr 23, 2024, 12:31 AM Billy Zha ***@***.***> wrote: What is the version of your ORAS CLI oras v1.2.0 What would you like to be added? Had an offline discussion with @shizhMSFT <https://github.com/shizhMSFT> and he suggests to add two commands: - oras snapshot (alias: dump, backup, pull-image) to support saving an image from remote registry to an OCI image layout. Usage: oras snapshot [flags] ***@***.***} <oci_layout_path>[:<tag>] - oras restore (alias: push-image) to support pushing an image from an OCI image layout to remote registry. Usage: oras restore [flags] ***@***.***} <reg>/<repo>[:<tag>] Why is this needed for ORAS? oras cp are not intuitive to uses, e.g. user might use oras push to push images from an OCI layout and succeed with false positive (see #1353 <#1353>). Adding those two wrapper commands of oras cp can help avoid that. Are you willing to submit PRs to contribute to this feature? - Yes, I am willing to implement it. — Reply to this email directly, view it on GitHub <#1366>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZNMMXDTQXO6V6M5ZEACLY6X52ZAVCNFSM6AAAAABGUFNS5WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI2TQMBVG44TCMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

shizhMSFT · 2024-04-24T03:24:20Z

Why not backup and restore?

Nice input. I've revised the proposal accordingly to oras backup and marked snapshot as an alias.

sajayantony · 2024-07-24T00:30:05Z

Could you also include reasoning why pull doesn't work here.

oras pull --to-oci-layout <oci_layout_path> <reg>/<repo>[{:<tag>|@digest]}] [...]

sajayantony · 2024-07-24T00:47:44Z

As per the call I'm ok if we limit to tar archive only dealing.

shizhMSFT · 2024-07-24T01:44:25Z

As per the call, here is an alternative without introducing new commands.

oras backup --> oras pull --archive <tar_path> <name>{:<tag>|@<digest>}
oras restore --> oras push --archive <tar_path> <name>[:<tag>[,<tag>][...]] <file>[:<type>] [...]

The major difference between oras backup/restore and oras pull/push is that oras backup/restore can take multiple images across registries and repositories.

Example usages are

# working with remote registry
oras pull localhost:5000/hello:v1 --archive hello.tar # backup hello:v1 to hello.tar
oras push localhost:5000/hello:v1 --archive hello.tar # restore hello:v1 from hello.tar

# working with entire repository
oras pull localhost:5000/hello --archive hello.tar # backup all images in hello to hello.tar
oras push localhost:5000/hello --archive hello.tar # restore all images in hello.tar

# working with multiple images
oras pull localhost:5000/hello:v1,foo,bar --archive hello.tar # backup v1, foo, and bar
oras push localhost:5000/hello:v1,foo --archive hello.tar     # restore v1 and foo, and ignore bar

# working with referrers
oras pull --include-referrers localhost:5000/hello:v1 --archive hello.tar
oras push --include-referrers localhost:5000/hello:v1 --archive hello.tar

# working with OCI layout. UX would be better if using `oras cp`.
oras pull --oci-layout layout.tar:v1 --archive hello.tar
oras pull --oci-layout layout-dir:v1 --archive hello.tar
oras push --oci-layout layout-dir:v1 --archive hello.tar

qweeah · 2024-07-24T02:40:05Z

oras restore --> oras push --archive <tar_path> [:[,][...]] [:] [...]

I think the usage is change to oras push <name>[:<tag>[,<tag>][...]] {<file>[:<type>] [...]|--archive <tar_path>}.

TerryHowe · 2024-07-24T10:44:14Z

Currently, pull skips successors though, right?

qweeah · 2024-07-24T13:09:37Z

Currently, pull skips successors though, right?

Depends on the type. subject are skipped unless --include-subject is specified. manifests in an index will be pulled.

TerryHowe · 2024-10-05T17:53:27Z

This issue is what I was requesting on #730

shizhMSFT · 2024-10-06T15:39:48Z

This issue is what I was requesting on #730

Yes, exactly. The difference is that the requirements are getting clearer after 1.5 years where more tools, including docker, are adopting OCI image layout.

liamwh · 2024-10-07T14:38:18Z

Is this being worked on at all? Would love to see it in ORAS!

qweeah · 2024-10-08T06:16:48Z

There is a little difference between the proposal and #730: cross-repository operation is not in scope here.

liamwh · 2024-10-09T06:45:17Z

Please forgive me @qweeah, I'm not sure I follow.

For my use case I just want to be able to pull and image and save it to disk, then be able to push it again to a different registry later. Is this being worked on and supported?

qweeah · 2024-10-09T07:01:46Z

Please forgive me @qweeah, I'm not sure I follow.

For my use case I just want to be able to pull and image and save it to disk, then be able to push it again to a different registry later. Is this being worked on and supported?

@liamwh Yes, it is already supported. You can use oras cp with --to-oci-layout(pull) and --from-oci-layout(push)

> # pull an image to local folder
> oras cp ghcr.io/oras-project/oras:v1.2.0 --to-oci-layout test:v1.2.0
✓ Copied  application/vnd.oci.image.config.v1+json                                                        1.56/1.56 kB 100.00%  626µs
  └─ sha256:9cd577b9700fa49ac6b2b8eca69731494a6212c7c7fca50669601e8068acc420
...
Copied [registry] ghcr.io/oras-project/oras:v1.2.0 => [oci-layout] test:v1.2.0
Digest: sha256:0087224dd0decc354b5b0689068fbbc40cd5dc3dbf65fcb3868dfbd363dc790b

An OCI image layout folder test will be created and contains the pulled image tagged v1.2.0

> tree .
.
└── test
    ├── blobs
    │   └── sha256
    │       ├── 0087224dd0decc354b5b0689068fbbc40cd5dc3dbf65fcb3868dfbd363dc790b
    ...
    ├── index.json
    ├── ingest
    └── oci-layout

You can push the image from the folder to your own registry like:

> # push an image from local folder
> oras cp --from-oci-layout test:v1.2.0 localhost:5000/oras:v1.2.0
✓ Copied  application/vnd.oci.image.config.v1+json                                                        1.57/1.57 kB 100.00%   86ms
  └─ sha256:306409c919f25a0c7900c4610a925b53f6fb32a251c0ec81ab8b4381296b0cf3
...
  └─ sha256:0087224dd0decc354b5b0689068fbbc40cd5dc3dbf65fcb3868dfbd363dc790b
Copied [oci-layout] test:v1.2.0 => [registry] localhost:5000/oras:v1.2.0
Digest: sha256:0087224dd0decc354b5b0689068fbbc40cd5dc3dbf65fcb3868dfbd363dc790b

liamwh · 2024-10-09T07:04:06Z

Ah ok great, thank you 🙏

Is it then correct to assume that this is also supported from the Go SDK?

liamwh · 2024-10-09T07:04:38Z

And is it possible to save the file as a tar instead of unpacking it?

qweeah · 2024-10-09T07:05:36Z

Saving(pulling) to a tar file is not supported, but reading(pushing) from a tar file is supported. You may tar the folder after pulling.

liamwh · 2024-10-09T08:47:04Z

Thanks for clarifying. Is it possible to save to a tar file instead of unpacking the contents to the file system?

TerryHowe · 2024-10-09T14:24:17Z

Is it possible to save to a tar file instead of unpacking the contents to the file system?

No

liamwh · 2024-10-14T09:55:00Z

Please forgive my ignorance, I'm just trying to understand: why not skip the extraction and support just downloading the tar, given that's what an OCI image actually is?

TerryHowe · 2024-10-14T10:42:00Z

Please forgive my ignorance, I'm just trying to understand: why not skip the extraction and support just downloading the tar, given that's what an OCI image actually is?

My understanding is that is exactly what this issue is about.

qweeah · 2024-10-14T11:42:59Z

why not skip the extraction and support just downloading the tar, given that's what an OCI image actually is?

@liamwh Image layer files can be tarball and oras will not extract layer files.

However, the image itself, consisting of layer tarball files and other metadata, is not a tarball. The OCI layout folder won't be compressed by oras.

qweeah added enhancement New feature or request triage New issues or PRs to be acknowledged by maintainers labels Apr 23, 2024

qweeah removed the triage New issues or PRs to be acknowledged by maintainers label May 7, 2024

qweeah added this to the v1.3.0 milestone May 7, 2024

shizhMSFT mentioned this issue Jul 24, 2024

Support pulling an OCI artifact/image from registry to a local OCI archive #1160

Open

1 task

shizhMSFT added the spec required Issues that require specifications label Jul 24, 2024

shizhMSFT mentioned this issue Oct 6, 2024

Add ability to download/upload one or more resources to/from disk #730

Open

1 task

FeynmanZhou assigned TerryHowe Nov 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new commands to support uploading and downloading images #1366

Add new commands to support uploading and downloading images #1366

qweeah commented Apr 23, 2024 •

edited

Loading

TerryHowe commented Apr 23, 2024 via email

shizhMSFT commented Apr 24, 2024

sajayantony commented Jul 24, 2024

sajayantony commented Jul 24, 2024

shizhMSFT commented Jul 24, 2024 •

edited

Loading

qweeah commented Jul 24, 2024 •

edited

Loading

TerryHowe commented Jul 24, 2024

qweeah commented Jul 24, 2024

TerryHowe commented Oct 5, 2024

shizhMSFT commented Oct 6, 2024

liamwh commented Oct 7, 2024

qweeah commented Oct 8, 2024

liamwh commented Oct 9, 2024

qweeah commented Oct 9, 2024

liamwh commented Oct 9, 2024

liamwh commented Oct 9, 2024

qweeah commented Oct 9, 2024 •

edited

Loading

liamwh commented Oct 9, 2024

TerryHowe commented Oct 9, 2024

liamwh commented Oct 14, 2024

TerryHowe commented Oct 14, 2024

qweeah commented Oct 14, 2024

Add new commands to support uploading and downloading images #1366

Add new commands to support uploading and downloading images #1366

Comments

qweeah commented Apr 23, 2024 • edited Loading

What is the version of your ORAS CLI

What would you like to be added?

Why is this needed for ORAS?

Are you willing to submit PRs to contribute to this feature?

TerryHowe commented Apr 23, 2024 via email

shizhMSFT commented Apr 24, 2024

sajayantony commented Jul 24, 2024

sajayantony commented Jul 24, 2024

shizhMSFT commented Jul 24, 2024 • edited Loading

qweeah commented Jul 24, 2024 • edited Loading

TerryHowe commented Jul 24, 2024

qweeah commented Jul 24, 2024

TerryHowe commented Oct 5, 2024

shizhMSFT commented Oct 6, 2024

liamwh commented Oct 7, 2024

qweeah commented Oct 8, 2024

liamwh commented Oct 9, 2024

qweeah commented Oct 9, 2024

liamwh commented Oct 9, 2024

liamwh commented Oct 9, 2024

qweeah commented Oct 9, 2024 • edited Loading

liamwh commented Oct 9, 2024

TerryHowe commented Oct 9, 2024

liamwh commented Oct 14, 2024

TerryHowe commented Oct 14, 2024

qweeah commented Oct 14, 2024

qweeah commented Apr 23, 2024 •

edited

Loading

shizhMSFT commented Jul 24, 2024 •

edited

Loading

qweeah commented Jul 24, 2024 •

edited

Loading

qweeah commented Oct 9, 2024 •

edited

Loading