Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCI layout documentation #574

Merged
merged 15 commits into from
Aug 18, 2023
Merged

OCI layout documentation #574

merged 15 commits into from
Aug 18, 2023

Conversation

jjbustamante
Copy link
Member

@jjbustamante jjbustamante commented Mar 6, 2023

The following pull requests adds documentation on how to use the new experimental feature to export the application image to disk in OCI layout format

I added a new section Experimental under the Feature section a it looks like this:

Screenshot 2023-08-17 at 8 23 33 AM

fixes #529

jjbustamante and others added 6 commits March 6, 2023 16:21
Signed-off-by: Juan Bustamante <[email protected]>
- Ruby 3+ no longer includes webrick so it has to be
explicitly required in the Gemfile for the app to start
- See: https://bugs.ruby-lang.org/issues/17303

Signed-off-by: Tim Downey <[email protected]>
@jjbustamante jjbustamante marked this pull request as ready for review March 27, 2023 22:12
@jjbustamante jjbustamante requested a review from a team as a code owner March 27, 2023 22:12
Copy link
Member

@AidanDelaney AidanDelaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great. I wonder if we should push most of our docs about "experimental" features to a new section? It is the case that only advanced users would be interested in these experimental features.

</div>

Exporting to OCI layout format is an **experimental** feature available on pack since version X.Y.Z

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a table somewhere comparing the cnb layout with the oci layout. The first question I'd ask as a user is "Which layout should I choose?", and such a table would contain enough "Why?" to answer which layout to choose.

```bash
cat ~/.pack/config.toml
experimental = true
layout-repo-dir = "<$HOME>/.pack/layout-repo"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does pack config experimental true add the "layout-repo-dir" to config.toml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's correct

content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
pack build oci:sample-app --path samples/apps/java-maven --builder cnbs/sample-builder:bionic
```

It will save the image in a folder `./sample-app` created in your current directory.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need layout-repo-dir if the OCI layout is saved to the PWD?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using the layout-repo-dir to save the run-image on disk because actually invoking the lifecycle. When lifecycle is configured to use OCI layout it is expecting to have all the require inputs (like the run-image) on disk, because that should be transparent for users, we are saving those images in that particular folder.

Comment on lines 89 to 90
Before using this option we suggest to remove your local layout directory (the one configured in your pack config.toml with the key `layout-repo-dir`) and
your application image folder (if you are planning to use the same folder), the reason for this is pack doesn't remove the blobs saved in the `layout-repo-dir` if you use the `--sparse` flag
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it becomes more unclear why layout-repo-dir was necessary above.

Comment on lines 80 to 81
If you are interested on keep playing with the image in OCI layout format, one tool you can take a look is [umoci](https://umo.ci/) it can help you to create a
[runtime bundler](https://github.com/opencontainers/runtime-spec) that can be executed with another tool like [runc](https://github.com/opencontainers/runc)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There a number of tools here that we might need to explain to end-users. I think we're using umoci because it is a strict-OCI compliant tool? That assembles an image. I presume that docker can also execute a strict-OCI image? If so, should we prefer to not introduce a new runtime there?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I had some doubts about including those tool, maybe we can suppose users know what they can do with the OCI layout format

Comment on lines 139 to 146
It will give you `application/vnd.oci.image.index.v1+json` media type, which is currently working

But [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane)

```bash
crane pull <your-image> <dest> --format=oci
```
It will give you `application/vnd.docker.distribution.manifest.list.v2+json`, which will fail
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these things pass/fail due to the state of our current implementation? Or because of a limitation in crane, or because the operation violates part of the OCI spec? Or for some other reason?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, helpful to add a little more detail here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It fails because of the state or our current implementation, it should be improve on new versions of the feature. I will add it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me explain this part a little better so you can guide me on how to document it.

According to the RFC if you want to export your application image on disk, you need to mount to the lifecycle a local storage with the images require by the build process, for example, the run-image. This storage must contain images in OCI layout format and name resolution will follow some rules defined also in the RFC.

The documented part was based on a feedback provided by @natalieparellano here, which is basically the following scenario: Suppose you want to execute the lifecycle directly without using pack, for example, a platform implementor. They need to create this local storage and save everything there before executing the build, right?, well the problem is: If they save in the local storage a manifest list and then they execute the lifecycle, right now, we are not smart enough to detect which manifest we need to pick to execute the build.

That is the actual comment we want to document so users can take it into consideration, now I am not sure this is the right place to do it, or maybe I didn't write it correctly, so any advice will be appreciate!

content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
content/docs/app-developer-guide/export-to-oci-layout.md Outdated Show resolved Hide resolved
Comment on lines 139 to 146
It will give you `application/vnd.oci.image.index.v1+json` media type, which is currently working

But [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane)

```bash
crane pull <your-image> <dest> --format=oci
```
It will give you `application/vnd.docker.distribution.manifest.list.v2+json`, which will fail
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, helpful to add a little more detail here

@jjbustamante
Copy link
Member Author

This is really great. I wonder if we should push most of our docs about "experimental" features to a new section? It is the case that only advanced users would be interested in these experimental features.

@AidanDelaney

Maybe I can move the documentation under Feature section, put something like Experimental there?

Screenshot 2023-05-05 at 3 20 54 PM

@natalieparellano
Copy link
Member

What's the status on this one? Does it still need changes?

Copy link
Member

@AidanDelaney AidanDelaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot. You raise a great point about the structure of our docs. I'll think about this.

@AidanDelaney AidanDelaney merged commit 35d15de into buildpacks:main Aug 18, 2023
github-actions bot pushed a commit that referenced this pull request Aug 18, 2023
…ent-export-to-oci-layout

OCI layout documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document Export to OCI layout feature
6 participants