Hermetic build documentation

antora-lunr-ui/css/main.css

@@ -109,4 +109,8 @@ i.fa.icon-caution::before {
 
 aside.toc.sidebar {
     overflow-block: scroll;
+}
+
+.image {
+    border: 1px solid #000000 !important;
 }

docs/modules/ROOT/nav-how-to-guides.adoc

@@ -3,6 +3,8 @@
 ** Configuring your builds
 *** xref:how-to-guides/configuring-builds/proc_upgrade_build_pipeline.adoc[Upgrading to a custom build pipeline]
 *** xref:how-to-guides/configuring-builds/proc_customize_build_pipeline.adoc[Customizing the pipeline] 
+*** xref:how-to-guides/proc_hermetic-builds.adoc[Enabling hermetic builds]
+*** xref:how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc[Prefetching package manager dependencies for hermetic build]
 ** Testing your application
 *** xref:how-to-guides/testing_applications/con_test-overview.adoc[Overview of {ProductName} tests]
 *** xref:how-to-guides/testing_applications/surface-level_tests.adoc[Surface-level tests]
@@ -14,7 +16,7 @@
 *** xref:how-to-guides/Secure-your-supply-chain/proc_java_dependencies.adoc[Configuring dependencies rebuild for Java apps in the CLI]
 ** xref:how-to-guides/proc_creating_your_own_environment.adoc[Creating your own environment]
 ** xref:how-to-guides/proc_managing-compliance-with-the-enterprise-contract.adoc[Managing compliance with the Enterprise Contract]
-** xref:how-to-guides/proc_delete_application.adoc[Deleting an application].
+** xref:how-to-guides/proc_delete_application.adoc[Deleting an application]
 
 
 ////

docs/modules/ROOT/pages/how-to-guides/proc_hermetic-builds.adoc

@@ -0,0 +1,54 @@
+:_content-type: PROCEDURE
+:myfunctionone: hermetic_builds
+
+[id="hermetic-builds_{context}"]
+= Enabling hermetic builds
+
+Hermetic builds refer to a controlled build process in which a build system knows all the required resources and dependencies. This enhanced dependency visibility allows the build system to capture and produce a more precise record of the build's origin, dependencies, and modifications than it would otherwise be possible.
+
+In {ProductName}, hermetic builds are achieved by blocking network access to the build process. Therefore, you must pre-fetch all dependencies before the build can start.
+
+IMPORTANT: Hermetic builds disable network access, so the builds that require dependencies outside its Git repository could fail.
+
+
+.Prerequisites
+
+* You have an link:https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/configuring-builds/proc_upgrade_build_pipeline/[upgraded build pipeline].
+
+.Procedure
+
+To create a hermetic build for a component, complete the following steps:
+
+. Go to the `.tekton` directory in the repository of your component and locate the *.yaml* files related to the *pull-request* and *push* processes.
+
+. Configure the hermetic pipeline by adding the following hermetic pipeline parameters in both the *.yaml* files:
+
++
+[source,yaml]
+----
+spec:
+    params:
+        -   ...
+        -   name: hermetic
+            value: “true”
+----
+
++
+[NOTE]
+====
+* In a non-java application, that is a link:https://github.com/redhat-appstudio/build-definitions/blob/main/task/buildah/0.1/buildah.yaml[Buildah] task, adding the parameter mentioned above automatically isolates the build from the network, restricting it to only build from dependencies listed in your Git repository. If you need to pull in dependencies from a package manager for one of the xref:how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc#supported-languages[supported languages], see xref:how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc[Prefetching the package manager dependencies for the Hermetic build].
+
+* When adding the parameters mentioned above, you can safely ignore the default values for the link:https://github.com/burrsutter/partner-catalog-stage/blob/e2ebb05ba8b4e842010710898d555ed3ba687329/.tekton/partner-catalog-stage-wgxd-pull-request.yaml#L87[`pipelineSpec.params`] in the *.yaml* files.
+====
+. Create a pull request (PR) by committing your changes to the repository of the component.
+
+. Review and merge the PR.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+[role="_additional-resources"]
+.Additional resources
+
+* For more information on the importance of provenance, see link:https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/concepts/slsa/con_slsa-conformity/[Supply chain security through SLSA conformity].
+
+* For more information on prefetching package manager dependencies, see the xref:how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc[Prefetching package manager dependencies for hermetic build]

docs/modules/ROOT/pages/how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc

@@ -0,0 +1,257 @@
+:_content-type: PROCEDURE
+:myfunctiontwo: prefetch_hermetic_builds
+
+[id="prefetching-dependencies-to-support-hermetic-build_{context}"]
+= Prefetching the package manager dependencies for the hermetic build
+
+In {ProductName}, hermetic builds are achieved by restricting network access to the build. Consequently, builds using supported languages may fail if they cannot retrieve dependencies from repositories. To address this, {ProductName} uses Cachi2 to prefetch the package manager dependencies for the supported-languages. Cachi2 makes it easier for package managers to prefetch dependencies. It also helps generate a manifest of all dependencies included in your builds, ensuring transparency and maintainability.
+
+[#supported-languages]
+.Supported languages
+
+[cols="1,1"]
+|===
+|Language
+|Package Manager
+
+|Go
+|`gomod`
+
+|Python
+|`pip`
+|===
+
+== Enabling prefetch builds for `gomod`
+
+.Prerequisites
+* You have an link:https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/configuring-builds/proc_upgrade_build_pipeline/[upgraded build pipeline].
+
+* You have a `go.mod` file in your repository that lists all the dependencies.
+
+.Procedures
+To create a hermetic build for a component, complete the following steps:
+
+. Go to the `.tekton` directory in the repository of your component and locate the *.yaml* files related to the *pull-request* and *push* processes.
+
+. Configure the hermetic pipeline by adding the following hermetic pipeline parameters in both the *.yaml* files:
+
++
+[source,yaml]
+----
+spec:
+    params:
+        -   ...
+        -   name: prefetch-input
+            value: '{"type": "gomod", "path": "."}' <1>
+----
+<1> The `prefetch-input` parameter specifies the package manager lock file relative to the package path. In the above example, the `.` indicates that the package manager lock file is located in the root of the repository. Additionally, if you have multiple lock files, you can provide the path to the lock file in the JSON array format. For example, `[{"path": ".", "type": "gomod"}, {"path": "subpath/to/the/other/repository", "type": "gomod"}]`.
+
++
+NOTE: When adding the parameters mentioned above, you can safely ignore the default values for the link:https://github.com/burrsutter/partner-catalog-stage/blob/e2ebb05ba8b4e842010710898d555ed3ba687329/.tekton/partner-catalog-stage-wgxd-pull-request.yaml#L90[`pipelineSpec.params`] in the *.yaml* files.
+
+. Create a pull request (PR) by committing your changes to the repository of the component.
+
+. Review and merge the PR.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+== Enabling prefetch builds for `pip`
+Cachi2 supports pip by parsing of `pip` requirements files, including but not limited to, *requirements.txt* files placed in the root of your repository. By generically parsing `pip` requirements files, Cachi2 downloads the specified dependencies.
+
+IMPORTANT: The requirements file can have a different name, as you can use multiple files to provide the dependencies. These requirements files function as lock files, encompassing all the transitive dependencies. You must actively pin each transitive dependency listed in the requirements file to a specific version.
+
+.Prerequisites
+* You have an link:https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/configuring-builds/proc_upgrade_build_pipeline/[upgraded build pipeline].
+
+* You have an environment that closely matches the environment in the container build, meaning it has the same operating system and the same python $major.$minor version.
+
+* You have installed the link:https://github.com/jazzband/pip-tools[pip-tools] package.
+
+.Procedures
+To create a hermetic build for a component, complete the following steps:
+
+. Go to your component’s source code.
+
+. Review your repository for a metadata file, for example, *pyproject.toml*, *setup.py*, or *setup.cfg*. If there is no metadata file, create one. Cachi2 looks for the name and version of your project in the metadata files. You can place this file at the root of your repository.
+
++
+[source,metadata]
+----
+[metadata]
+name = "my_package"
+version = "0.1.0"
+----
+
++
+NOTE: Instead of a *pyproject.toml* file, you can also create a *setup.py* or *setup.cfg* file. For information on the metadata of these files, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#project-metadata[Project Metadata].
+
+. Generate a fully resolved *requirements.txt* file that contains all the transitive dependencies and pins them to a specific version and hash by using the following command:
+
++
+[source,command]
+----
+$ pip-compile pyproject.toml --generate-hashes
+----
+
++
+[NOTE]
+==== 
+* The above-mentioned command assumes that you have defined project dependencies in *pyproject.toml*. However, if you have defined the project dependencies in either the *setup.py*, *requirements.txt*, or *requirements.in* files, ensure to update the command accordingly.
+====
+. Add the *requirements.txt* to the root of your component’s source code. 
+
+. Go to the `.tekton` directory and locate the *.yaml* files related to the *pull-request* and *push* processes.
+
+. Configure the hermetic pipeline by adding the following hermetic pipeline parameters in both the *.yaml* files:
+
++
+[source,yaml]
+----
+spec:
+    params:
+        -   ...
+        -   name: prefetch-input
+            value: '{"type": "pip", "path": "."}' <1>
+----
+<1> The `prefetch-input` parameter specifies the package manager lock file relative to the package path. In the above example, the `.` indicates that the package manager lock file is located in the root of the repository. Additionally, if you have multiple lock files, you can provide the path to the lock file in the JSON array format. For example, `[{"path": ".", "type": "pip"}, {"path": "subpath/to/the/other/repository", "type": "pip"}]`.
+
++
+NOTE: When adding the parameters mentioned above, you can safely ignore the default values for the link:https://github.com/burrsutter/partner-catalog-stage/blob/e2ebb05ba8b4e842010710898d555ed3ba687329/.tekton/partner-catalog-stage-wgxd-pull-request.yaml#L90[`pipelineSpec.params`] in the *.yaml* files.
+
+. Create a PR by committing your changes to the repository of the component.
+
+. Review and merge the PR.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+== Enabling prefetch builds for package dependencies in `pip`
+
+If your project has package dependencies in `pip`, you can prefetch builds to improve the performance of the build process. Prefetch builds allow Cachi2 to fetch the build dependencies and provide them to Pip for offline installation in a network-isolated environment.
+
+.Prerequisites
+
+* You have an link:https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/configuring-builds/proc_upgrade_build_pipeline/[upgraded build pipeline].
+
+* You have an environment that closely matches the environment in the container build, meaning it has the same operating system and the same python $major.$minor version.
+
+* You have installed the link:https://github.com/jazzband/pip-tools[pip-tools] package.
+
+.Procedures
+To create a hermetic build for a component, complete the following steps:
+
+. Download the link:https://raw.githubusercontent.com/containerbuildsystem/cachito/master/bin/pip_find_builddeps.py[`pip_find_builddeps.py`] script directly from GitHub.
+
++
+NOTE: This script has no runtime dependency other than `pip`.
+
+. Add the script that you downloaded in a directory that is already included in your $PATH. For example, you can use the `~/bin` directory in your home folder. Ensure that it exists or create it if needed. To add it to the $PATH permanently, you can modify the shell configuration file (for example, `.bashrc`, `.bash_profile`, or `.zshrc`)  and restart the terminal after appending the following line:
+
++
+[source,bash]
+----
+export PATH="$HOME/bin:$PATH"
+----
+. Open the terminal and go to the directory where you placed the `pip_find_builddeps.py` script and run the following command to make it executable:
+
++
+[source,bash]
+----
+chmod +x pip_find_builddeps.py
+----
+
+. Go to your component's source code.
+
+. Review the root of your repository for a metadata file, for example, *pyproject.toml*, *setup.py*, or *setup.cfg*. If there is no metadata file, create one, as Cachi2 looks for the name and version of your project in the metadata files.
+
++
+[source,metadata]
+----
+[metadata]
+name = "my_package"
+version = "0.1.0"
+----
+
++
+NOTE: Instead of a *pyproject.toml* file, you can also create a *setup.py* or *setup.cfg* file. For information on the metadata of these files, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#project-metadata[Project Metadata].
+
+. Generate a fully resolved *requirements.txt* file that contains all the transitive dependencies and pins them to a specific version and hash by using the following command:
+
++
+[source,command]
+----
+$ pip-compile pyproject.toml --generate-hashes
+----
+
++
+[NOTE]
+==== 
+* To successfully run the command mentioned above your environment must be as close as possible to the environment in the container build. That is, the environment should have the same operating system and the same Python $major.$minor version.
+
+* The above-mentioned command assumes that you have defined project dependencies in *pyproject.toml*. However, if you have defined the project dependencies in either the *setup.py*, *requirements.txt*, or *requirements.in* files, ensure to update the command accordingly.
+====
+
+. Add the *requirements.txt* to the root of your component’s source code. 
+
+. In the root of your repository create a file, *requirements-build.in*.
+
+. Copy the build system requirements from the *pyproject.toml* file to the *requirements-build.in* file.
+
++
+*For example:*
++
+image::build-requirements.png[Build requirements, role="image"]
+
+. Run the `pip_find_builddeps.py` script and `pip-compile` the outputs by using the following command:
+
++
+[source,command]
+----
+$ pip_find_builddeps.py requirements.txt \
+--append \
+--only-write-on-update \
+-o requirements-build.in
+----
+
+. Use the `pip-compile` command to convert the *requirements-build.in* file in to the *requirements-build.txt* file by using the following command:
+
++
+[source,command]
+----
+$ pip-compile requirements-build.in --allow-unsafe --generate-hashes
+----
+
+. Add the *requirement-build.txt* file to your project. It does not require any changes to your build process. 
+
++
+NOTE: `pip` automatically installs the build dependencies when needed for explicit installation. The purpose of the *requirement-build.txt* file is to enable Cachi2 to fetch the build dependencies and provide them to `pip` for offline installation in a network-isolated environment.
+
+. Go to the `.tekton` directory and locate the *.yaml* files related to the *pull-request* and *push* processes.
+
+. Configure the hermetic pipeline by adding the following hermetic pipeline parameters in both the *.yaml* files:
+
++
+[source,yaml]
+----
+spec:
+    params:
+        -   ...
+        -   name: prefetch-input
+            value: '{"type": "pip", "path": "."}' <1>
+----
+<1> The `prefetch-input` parameter specifies the package manager lock file relative to the package path. In the above example, the `.` indicates that the package manager lock file is located in the root of the repository. Additionally, if you have multiple lock files, you can provide the path to the lock file in the JSON array format. For example, `[{"path": ".", "type": "pip"}, {"path": "subpath/to/the/requirements/repository", "type": "pip"}]`.
+
++
+NOTE: When adding the parameters mentioned above, you can safely ignore the default values for the link:https://github.com/burrsutter/partner-catalog-stage/blob/e2ebb05ba8b4e842010710898d555ed3ba687329/.tekton/partner-catalog-stage-wgxd-pull-request.yaml#L90[`pipelineSpec.params`] in the *.yaml* files.
+
+. Create a PR by committing your changes to the repository of the component.
+
+. Review and merge the PR.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+[role="_additional-resources"]
+.Additional resources
+
+* To troubleshoot any issues you experience when working with Enabling prefetch builds for `pip` or `pip` with source dependencies, see the link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#troubleshooting[Troubleshooting] section.
+
+* For more information about Cachi2, see link:https://github.com/containerbuildsystem/cachi2/blob/main/README.md[Cachi2] documentation.

docs/modules/ROOT/pages/partials/con_hermetic_verification.adoc

@@ -0,0 +1,16 @@
+.Verification
+
+. On your application, go to the *Activity > Pipeline runs* tab.
+
+ifdef::myfunctionone[]
+
+. Go to the pipeline run with *Type* as *Build* and confirm the `build-container` stage displays a green check mark indicating that the build process has successfully fetched all the dependencies.
+
+endif::[]
+
+ifdef::myfunctiontwo[]
+
+. For a pipeline run with *Type* as *Build* and confirm the `pre-fetch dependencies` stage displays a green check mark indicating that the build process has successfully pre-fetched all the dependencies.
+
+endif::[]
+