Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Builder extension spec #193

Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ When the specification refers to a path in the context of an OCI layer tar (e.g.
- [Bindings Extension Specification](extensions/bindings.md)
- [Buildpack Registry Extension Specification](extensions/buildpack-registry.md)
- [Project Descriptor Extension Specification](extensions/project-descriptor.md)
- [Builder Extension Specification](extensions/builder.md)

## API Versions

Expand Down
3 changes: 3 additions & 0 deletions distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,6 @@ Each stack ID MUST only be present once.
For a given stack, the `mixins` list MUST enumerate mixins such that no included buildpacks are missing a mixin for the stack.

Fewer stack entries as well as additional mixins for a stack entry MAY be specified.

### Builders
Buildpacks may be distributed using builders, as described in the [builder extension spec](extensions/builder.md).
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
180 changes: 180 additions & 0 deletions extensions/builder.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Builder Specification <!-- omit in toc -->

This document specified the artifact format for a builder.
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

A builder is an OCI image that provides a distributable build environment.

A [platform][platform-spec] supporting the builder extension specification SHOULD allow users to configure the build environment with a provided builder.

## Table of Contents <!-- omit in toc -->
- [General Requirements](#general-requirements)
- [Builder API Version](#builder-api-version)
- [File/Directories](#filedirectories)
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
- [Environment Variables](#environment-variables)
- [Labels](#labels)
- [Data Format](#data-format)
- [Labels](#labels-1)
- [`io.buildpacks.builder.metadata` (JSON)](#iobuildpacksbuildermetadata-json)
- [`io.buildpacks.buildpack.order` (JSON)](#iobuildpacksbuildpackorder-json)
- [`io.buildpacks.buildpack.layers` (JSON)](#iobuildpacksbuildpacklayers-json)
- [`io.buildpacks.lifecycle.apis` (JSON)](#iobuildpackslifecycleapis-json)

## General Requirements
A **builder** is an image that MUST contain an implementation of the lifecycle, and build-time environment, and MAY contain buildpacks. Platforms SHOULD use builders to ease the build process.
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
hone marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wording here still feels a bit awkward to me. However it also seems like a repetition of the first section. Maybe instead of word smithing we should just remove it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you dislike the first sentence?

I do think the second sentence the "SHOULD" probably is a "MAY".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a suggestion

dfreilich marked this conversation as resolved.
Show resolved Hide resolved

### Builder API Version
This document specifies Builder API version `0.1`.

dfreilich marked this conversation as resolved.
Show resolved Hide resolved
Builder API versions:
- MUST be in form `<major>.<minor>` or `<major>`, where `<major>` is equivalent to `<major>.0`
- When `<major>` is greater than `0` increments to `<minor>` SHALL exclusively indicate additive changes

### File/Directories
A builder MUST have the following directories/files:
- `/cnb/order.toml` &rarr; As defined in the [platform specification][order-toml-spec]
- `/cnb/stack.toml` &rarr; As defined in the [platform specification][stack-toml-spec]
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
- `/cnb/lifecycle/<lifecycle binaries>` &rarr; An implementation of the lifecycle, which contains the required lifecycle binaries for [building images][lifecycle-for-build].

In addition, every buildpack blob contained on a builder MUST be stored at the following file path:
- `/cnb/buildpacks/...<buildpack ID>/<buildpack version>/`
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

If the buildpack ID contains a `/`, it MUST be replaced with `_` in the directory name.

All specified files and directories are writeable by the build environment's User.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want buildpacks to have permission to change the order.toml or the code of other buildpacks? The files above are all currently root owned in builders created by pack.

We do want the layers dir and app dir to be writable by the build user however and should probably call that out explicitly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume that it definitely shouldn't be able to change the code of other buildpacks (except perhaps stackpacks).

So should I change this to something like:

The `CNB_APP_DIR` and `CNB_LAYERS_DIR` are writeable by the build environment's User.

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a suggestion...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
All specified files and directories are writeable by the build environment's User.
The `CNB_APP_DIR` and `CNB_LAYERS_DIR` SHOULD be writeable by the build environment's User.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested as SHOULD since I don't think it's required (the lifecycle will change permissions on those directories). Maybe we want it to be a MUST though?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to get the lifecycle out of the chowning business in my opinion, which we could do if it was gauranteed that this directory is writable by the build user.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that something we want to do right now? Current builders don't have that requirement.


### Environment Variables
A builder MUST be an extension of a build-image, and MUST retain all specified environment variables and labels set on the original build image, as specified in the [platform specifications][build-image-specs].
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

The following environment variables MUST be set on the builder:

| Env Variable | Description |
| ------------------ | --------------------------------------------------------------------------------------------------- |
| `CNB_APP_DIR` | Application directory of the build environment (eg: `/workspace`) |
| `CNB_LAYERS_DIR` | The directory to create and store `layers` in the build environment (eg: `/layers`) |
| `CNB_PLATFORM_DIR` | The directory to create and store platform focused files in the build environment (eg: `/platform`) |
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

The following variables MAY be set in the builder environment (through the image config's `Env` field):
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

| Env Variable | Description |
| ---------------------- | -------------------------------------- |
| `SERVICE_BINDING_ROOT` | The directory where services are bound |


### Labels
The following labels MUST be set in the builder environment (through the image config's `Labels` field):
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

| Label | Description |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `io.buildpacks.builder.api` | The [Builder API Version](#builder-api-version) this builder implements |
| `io.buildpacks.builder.metadata` | A JSON object representing [Builder Metadata](#iobuildpacksbuildermetadata) |
| `io.buildpacks.buildpack.order` | A JSON object representing the [order of the buildpacks stored on the builder](#iobuildpacksbuildpackorder) |
| `io.buildpacks.buildpack.layers` | A JSON object representing the [buildpack layers](#iobuildpacksbuildpacklayers) |
| `io.buildpacks.lifecycle.version` | A string, representing the version of the lifecycle stored in the builder |
| `io.buildpacks.lifecycle.apis` | A JSON object representing the [lifecycle APIs](#iobuildpackslifecycleapis) the lifecycle stored in the builder supports |

## Data Format
### Labels
#### `io.buildpacks.builder.metadata` (JSON)

```json
{
"description": "<description>",
"stack": {
"runImage": {
"image": "<run-image>",
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
"mirrors": [
"<run-image-mirror>"
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
]
}
},
"buildpacks": [
{
"id": "<buildpack ID>",
"version": "<buildpack version>",
"homepage": "<buildpack homepage>"
}
],
"createdBy": {
"name": "<tool name>",
"version": "<tool version>"
}
}
```

The `createdBy` metadata is meant to contain the name and version of the tool that created the builder.

#### `io.buildpacks.buildpack.order` (JSON)

```json
[
{
"group":
[
{
"id": "<buildpack ID>",
"version": "<buildpack version>",
"optional": false
}
]
}
]
```

#### `io.buildpacks.buildpack.layers` (JSON)
dfreilich marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"<buildpack ID>": {
"<buildpack version>": {
"api": "<buildpack API>",
"order": [
{
"group": [
{
"id": "<inner-buildpack>",
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
"version": "<inner-buildpack version>"
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
}
]
}
],
"layerDiffID": "<diff-ID>",
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
"homepage": "<buildpack homepage>"
}
},
"<inner buildpack>": {
"<inner buildpacks version>": {
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
"api": "<buildpack API>",
"stacks": [
{
"id": "<stack ID buildpack supports>",
"mixins": ["<list of mixins required>"]
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
}
],
"layerDiffID": "<diff-ID>",
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
"homepage": "<buildpack-homepage>"
dfreilich marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
```

#### `io.buildpacks.lifecycle.apis` (JSON)

```json
{
"buildpack": {
"deprecated": ["<list of versions>"],
"supported": ["<list of versions>"]
},
"platform": {
"deprecated": ["<list of versions>"],
"supported": ["<list of versions>"]
}
}
```

[//]: <> (Links)
[build-image-specs]: https://github.com/buildpacks/spec/blob/main/platform.md#build-image
[platform-spec]: https://github.com/buildpacks/spec/blob/main/platform.md
[order-toml-spec]: https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml
[stack-toml-spec]: https://github.com/buildpacks/spec/blob/main/platform.md#stacktoml-toml
[lifecycle-for-build]: https://github.com/buildpacks/spec/blob/main/platform.md#build
2 changes: 2 additions & 0 deletions platform.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ The platform SHOULD ensure that:
- The image config's `Label` field has the label `io.buildpacks.stack.description` set to the description of the stack.
- The image config's `Label` field has the label `io.buildpacks.stack.metadata` set to additional metadata related to the stack.

A [builder](extensions/builder.md) MUST be an extension of a build image.

### Run Image

The platform MUST ensure that:
Expand Down