Add spec changes for image extensions

Signed-off-by: Natalie Arellano <[email protected]>

buildpack.md

@@ -4,17 +4,21 @@ This document specifies the interface between a lifecycle program and one or mor
 
 The lifecycle program uses buildpacks to build software artifacts from source code and pack the result into an OCI image.
 
-This is accomplished in four phases:
+This is accomplished in six phases:
 
-1. **Detection,** where an optimal selection of compatible buildpacks is chosen.
-2. **Analysis,** where metadata about OCI layers generated during a previous build are made available to buildpacks.
-3. **Build,** where buildpacks use that metadata to generate only the OCI layers that need to be replaced.
-4. **Export,** where the remote layers are replaced by the generated layers.
+1. **Analysis**, where metadata about OCI layers generated during a previous build are made available to buildpacks.
+2. **Detection**, where an optimal selection of compatible buildpacks is chosen.
+3. **Build-Ext** (optional, see the [Image Extension Interface](image-extension.md)), where image extensions generate Dockerfiles that can be used to extend the build and/or run base images.
+4. **Extension** (optional, see the [Image Extension Interface](image-extension.md)), where Dockerfiles generated by image extensions are applied to the build and/or run base images.
+5. **Build**, where buildpacks use analyzed metadata to generate only the OCI layers that need to be replaced.
+6. **Export**, where the remote layers are replaced by the generated layers.
 
 The `ENTRYPOINT` of the OCI image contains logic implemented by the lifecycle that executes during the **Launch** phase.
 
 ## Table of Contents
 
+TODO: fix toc
+
 <!-- Using https://github.com/yzhang-gh/vscode-markdown to manage toc -->
 - [Buildpack Interface Specification](#buildpack-interface-specification)
   - [Table of Contents](#table-of-contents)
@@ -38,6 +42,9 @@ The `ENTRYPOINT` of the OCI image contains logic implemented by the lifecycle th
   - [Phase #2: Analysis](#phase-2-analysis)
     - [Purpose](#purpose-1)
     - [Process](#process-1)
+  - [Phase #3: Image Extension](#phase-3-image-extension)
+    - [Purpose](#purpose)
+    - [Process](#process)
   - [Phase #3: Build](#phase-3-build)
     - [Purpose](#purpose-2)
     - [Process](#process-2)
@@ -295,23 +302,27 @@ At the end of each individual buildpack's build phase:
 | `<app>/.profile`       | [†](README.md#linux-only) Bash-formatted script sourced by shell before launch
 | `<app>/.profile.bat`   | [‡](README.md#windows-only) BAT-formatted script sourced by shell before launch
 
+TODO: move this after analyze
+
 ## Phase #1: Detection
 
 ![Detection](img/detection.svg)
 
 ### Purpose
 
-The purpose of detection is to find an ordered group of buildpacks to use during the build phase.
-These buildpacks must be compatible with the app.
+The purpose of detection is to find an ordered group of image extensions and buildpacks to use during the build phase.
+These image extensions and buildpacks must be compatible with the app.
+
+For detect requirements that are specific to image extensions, see the [Image Extension Interface](image-extension.md).
 
 ### Process
 
 **GIVEN:**
-- An ordered list of buildpack groups resolved into buildpack implementations as described in [Order Resolution](#order-resolution)
+- An ordered list of image extension or buildpack groups resolved into buildpack implementations as described in [Order Resolution](#order-resolution)
 - A directory containing application source code
 - A shell, if needed,
 
-For each buildpack in each group in order, the lifecycle MUST execute `/bin/detect`.
+For each buildpack implementation in each group in order, the lifecycle MUST execute `/bin/detect`. Image extensions and buildpacks are both buildpack implementations. For simplicity, the following section will refer to both as "buildpacks". Image extensions MUST always be optional during detection.
 
 1. **If** the exit status of `/bin/detect` is non-zero and the buildpack is not marked optional, \
    **Then** the lifecycle MUST proceed to the next group or fail detection completely if no more groups are present.
@@ -325,7 +336,7 @@ For each buildpack in each group in order, the lifecycle MUST execute `/bin/dete
          **Then** the lifecycle MUST proceed to the next group or fail detection completely if no more groups are present.
 
       2. **If** at least one exit status from `/bin/detect` in the group is zero \
-         **Then** the lifecycle MUST select this group and proceed to the analysis phase.
+         **Then** the lifecycle MUST select this group and proceed to the builder-ext phase.
 
 The selected group MUST be filtered to only include buildpacks with exit status zero.
 The order of the buildpacks in the group MUST otherwise be preserved.
@@ -373,12 +384,14 @@ A buildpack's mixin requirements must be satisfied by the stack in one of the fo
 
 #### Order Resolution
 
-During detection, an order definition MUST be resolved into individual buildpack implementations.
+During detection, an order definition for image extensions (if present) and an order definition for buildpacks MUST be resolved into individual buildpack implementations.
+
+Order definitions for image extensions MUST NOT contain nested orders. If an order definition for image extensions is present, it will be pre-pended to the order definition for buildpacks (as if the order definition for image extensions were an order buildpack).
 
 The resolution process MUST follow this pattern:
 
 Where:
-- O and P are buildpack orders.
+- O and P are image extension or buildpack orders.
 - A through H are buildpack implementations.
 
 Given:
@@ -456,7 +469,104 @@ For each buildpack in the group, the lifecycle
 
 After analysis, the lifecycle MUST proceed to the build phase.
 
-## Phase #3: Build
+## Phase #3: Build-Ext (optional)
+
+### Purpose
+
+The purpose of the build-ext phase is to generate Dockerfiles that can be used to extend the build and/or run base images. The Build-Ext phase MUST NOT be run for Windows builds.
+
+### Process
+
+**GIVEN:**
+- The final ordered group of image extensions determined during the detection phase,
+- A directory containing application source code,
+- The Buildpack Plan,
+- An `<output>` directory used to store generated artifacts
+- A shell, if needed,
+
+For each image extension in the group in order, the lifecycle MUST execute `/bin/build`.
+
+1. **If** the exit status of `/bin/build` is non-zero, \
+   **Then** the lifecycle MUST fail the build.
+
+2. **If** the exit status of `/bin/build` is zero,
+   1. **If** there are additional image extensions in the group, \
+      **Then** the lifecycle MUST proceed to the next image extension's `/bin/build`.
+
+   2. **If** there are no additional image extensions in the group, \
+      **Then** the lifecycle MUST proceed to the extend phase.
+
+For each `/bin/build` executable in each image extension, the lifecycle:
+
+- MUST provide path arguments to `/bin/build` as described in the [Image Extension Interface](image-extension.md) section.
+- MUST configure the build environment as described in the [Environment](#environment) section.
+- MUST provide all `<plan>` entries that were required by any image extension or buildpack in the group during the detection phase with names matching the names that the image extension provided.
+
+Correspondingly, each `/bin/build` executable:
+
+- MAY read from the `<app>` directory.
+- MUST NOT write to the `<app>` directory.
+- MAY read the build environment as described in the [Environment](#environment) section.
+- MAY read the Buildpack Plan.
+- MAY log output from the build process to `stdout`.
+- MAY emit error, warning, or debug messages to `stderr`.
+- MAY write any combination of Dockerfile, build.Dockerfile, and run.Dockerfile to the `<output>` directory. These files MUST adhere to the requirements listed below.
+- MAY write key-value pairs to `<output>/launch.toml` that are provided as build args to Dockerfile or run.Dockerfile when extending the run image.
+- MAY write key-value pairs to `<output>/build.toml` that are provided as build args to Dockerfile or build.Dockerfile when extending the build image.
+- SHOULD write SBOM (Software-Bill-of-Materials) files as described in the [Software-Bill-of-Materials](#software-bill-of-materials) section describing any contributions to the app image or build environment.
+
+#### Dockerfile Requirements
+
+run.Dockerfiles have the following requirements:
+
+- TODO: list run.Dockerfile requirements
+- Dockerfiles that target the run image (only) may indicate any image in their `FROM` instruction.
+
+Dockerfiles and build.Dockerfiles have the following additional requirements:
+
+- TODO: list build.Dockerfile requirements
+
+## Phase #4: Extension
+
+### Purpose
+
+The purpose of the extension phase is to apply the Dockerfiles generated in the build-ext phase to the build and/or run base images. The Extension phase MUST NOT be run for Windows builds.
+
+### Process
+
+The extension phase MUST be invoked separately for each base image (build or run) that is extended.
+
+**GIVEN:**
+- The final ordered group of Dockerfiles generated during the build-ext phase,
+- Buildpack-provided build args specified during the build-ext phase,
+- A UUID provided as a `build_id` build arg,
+- A user id provided as a `user_id` build arg, and a group id provided as a `group_id` build arg,
+- A Dockerfile build context provided as a `base_image` build arg,
+
+The `build_id` build arg allows the Dockerfile to invalidate the cache after a certain layer. When the `$build_id` build arg is referenced in a `RUN` instruction, all subsequent layers will be rebuilt on the next build (as the value will change).
+
+The `user_id` and `group_id` build args allow the Dockerfile to reset the `USER` to its original value if the user needs to be changed (e.g., to `root`) in order to perform privileged operations.
+
+For each Dockerfile in the group in order, the lifecycle MUST apply the Dockerfile to the base image as follows:
+
+- For the first Dockerfile, the `base_image` build arg MUST be the original base image.
+- When there are multiple Dockerfiles, the intermediate image generated from the application of the current Dockerfile MUST be provided as the `base_image` build arg to the next Dockerfile.
+
+#### Ability to rebase
+
+When extending the run image, a Dockerfile MAY indicate that it preserves the ability to rebase by including the following instruction:
+
+```bash
+LABEL io.buildpacks.rebasable=true
+```
+
+For the final image to be rebasable, the provided run image and all applied Dockerfiles MUST indicate rebasability.
+
+#### Software-Bill-of-Materials
+
+Image extensions MAY write Software Bill of Materials (SBOM) files as described in [Software-Bill-of-Materials](#software-bill-of-materials). The platform MAY generate additional SBOM files describing an extended base image after all Dockerfiles have been applied.
+
+## Phase #5: Build
 
 ![Build](img/build.svg)
 
@@ -544,14 +654,14 @@ Correspondingly, each `/bin/build` executable:
 
 #### Unmet Buildpack Plan Entries
 
-A buildpack SHOULD designate a Buildpack Plan entry as unmet if the buildpack did not satisfy the requirement described by the entry.
+An image extension or buildpack SHOULD designate a Buildpack Plan entry as unmet if the buildpack did not satisfy the requirement described by the entry.
 The lifecycle SHALL assume that all entries in the Buildpack Plan were satisfied by the buildpack unless the buildpack writes an entry with the given name to the `unmet` section of `build.toml`.
 
 For each entry in `<plan>`:
   - **If** there is an unmet entry in `build.toml` with a matching `name`, the lifecycle
-    - MUST include the entry in the `<plan>` of the next buildpack that provided an entry with that name during the detection phase.
+    - MUST include the entry in the `<plan>` of the next image extension or buildpack that provided an entry with that name during the detection phase.
   - **Else**, the lifecycle
-    - MUST NOT include entries with matching names in the `<plan>` provided to subsequent buildpacks.
+    - MUST NOT include entries with matching names in the `<plan>` provided to subsequent image extensions or buildpacks.
 
 #### Software-Bill-of-Materials
 
@@ -607,7 +717,7 @@ Additionally, a buildpack MAY specify sub-paths within `<app>` as `slices` in `l
 Separate layers MUST be created during the export phase for each slice with one or more files or directories.
 This minimizes data transfer when the app directory contains a known set of files.
 
-## Phase #4: Export
+## Phase #6: Export
 
 ![Export](img/export.svg)
 
@@ -882,12 +992,12 @@ Prohibited:
 
 ### Requirements
 
-The lifecycle MUST be implemented so that the detection and build phases do not have access to OCI image store credentials used in the analysis and export phases.
+The lifecycle MUST be implemented so that the detection, build-ext, extension, and build phases do not have access to OCI image store credentials used in the analysis and export phases.
 The lifecycle SHOULD be implemented so that each phase may run in a different container.
 
 ## Data Format
 
-### launch.toml (TOML)
+### launch.toml (TOML) for buildpacks; for image extensions see the [Image Extension Specification](image-extension.md)
 
 ```toml
 [[labels]]
@@ -948,7 +1058,7 @@ The lifecycle MUST accept slices that do not contain any files or directory. How
 
 The lifecycle MUST include all unmatched files in the app directory in any number of additional layers in the OCI image.
 
-### build.toml (TOML)
+### build.toml (TOML) for buildpacks; for image extensions see the [Image Extension Specification](image-extension.md)
 
 ```toml
 [[unmet]]
@@ -1104,7 +1214,7 @@ Each stack in `stacks` either:
 
 #### Order Buildpacks
 
-A buildpack descriptor that specifies `order` MUST be [resolvable](#order-resolution) into an ordering of buildpacks that implement the [Buildpack Interface](#buildpack-interface).
+A buildpack descriptor that specifies `order` MUST be [resolvable](#order-resolution) into an ordering of buildpacks that implement the [Buildpack Interface](#buildpack-interface). The `order` MUST include only regular buildpacks and MUST NOT include image extensions.
 
 A buildpack reference inside of a `group` MUST contain an `id` and `version`.
 

image-extension.md

@@ -0,0 +1,101 @@
+# Image Extension Interface Specification
+
+This document specifies the interface between a lifecycle program and one or more image extensions.
+
+The lifecycle program uses image extensions to generate Dockerfiles that can be used to extend build and/or run base images prior to buildpacks builds.
+
+## Table of Contents
+
+<!-- Using https://github.com/yzhang-gh/vscode-markdown to manage toc -->
+
+## Image Extension API Version
+
+This document accompanies Buildpack API version `0.8`.
+
+## Image Extension Interface
+
+Unless otherwise noted, image extensions are expected to conform to the [Buildpack Interface Specification](buildpack.md).
+
+### Detection
+
+Executable: `/bin/detect`, Working Dir: `<app[AR]>`
+
+Image extensions participate in the buildpack [detection](buildpack.md#detection) process, with the same interface for `/bin/detect`. However:
+- Detection is optional for image extensions, and they are assumed to pass detection when `/bin/detect` is not present.
+- Image extensions MUST only output `provides` entries to the build plan. They MUST NOT output `requires`.
+
+### Build-Ext
+
+Executable: `/bin/build`, Working Dir: `<app[AR]>`
+
+Image extensions participate in a build-ext process that is similar to the buildpack [build](buildpack.md#build) process, with the same interface for `/bin/build`. However:
+- Image extensions' `/bin/build` MUST NOT write to the app directory.
+- `$CNB_LAYERS_DIR` MUST be the absolute path of an `<output>` directory and MUST NOT be the path of the buildpack layers directory.
+- If an image extension is missing `/bin/build`, the image extension root MUST be treated as a pre-populated `<output>` directory.
+
+## Data Format
+
+### Files
+
+### extension.toml (TOML)
+
+This section describes the 'Extension descriptor'.
+
+```toml
+api = "<buildpack API version>"
+
+[extension]
+id = "<extension ID>"
+name = "<extension name>"
+version = "<extension version>"
+homepage = "<extension homepage>"
+description = "<extension description>"
+keywords = [ "<string>" ]
+sbom-formats = [ "<string>" ]
+
+[[extension.licenses]]
+type = "<string>"
+uri = "<uri>"
+```
+
+Image extension authors MUST choose a globally unique ID, for example: "io.buildpacks.apt".
+
+The image extension `id`, `version`, `api`, `licenses`, and `sbom-formats` entries MUST follow the requirements defined in the [Buildpack Interface Specification](buildpack.md).
+
+### launch.toml (TOML) for extensions; for buildpacks see the [Buildpack Interface Specification](buildpack.md)
+
+```toml
+[[args]]
+name = "<build arg name>"
+value = "<build arg value>"
+
+[[labels]]
+key = "<label key>"
+value = "<label valu>"
+```
+
+The image extension MAY specify any number of args or labels.
+
+For each arg, the image extension:
+- MUST specify a `name` to be the name of a build argument that will be provided to any output Dockerfile or run.Dockerfile when extending the run base image.
+- MUST specify a `value` to be the value of the build argument when it is provided.
+
+The image extension `labels` entries MUST follow the requirements defined in the [Buildpack Interface Specification](buildpack.md).
+
+### build.toml (TOML) for buildpacks; for image extensions see the [Image Extension Specification](image-extension.md)
+
+```toml
+[[args]]
+name = "<build arg name>"
+value = "<build arg value>"
+
+[[unmet]]
+name = "<dependency name>"
+```
+
+For each arg, the image extension:
+
+- MUST specify a `name` to be the name of a build argument that will be provided to any output Dockerfile or build.Dockerfile when extending the build base image.
+- MUST specify a `value` to be the value of the build argument when it is provided.
+
+The image extension `unmet` entries MUST follow the requirements defined in the [Buildpack Interface Specification](buildpack.md).