Hermetic build documentation

antora-lunr-ui/css/main.css

@@ -109,4 +109,8 @@ i.fa.icon-caution::before {
 
 aside.toc.sidebar {
  overflow-block: scroll;
+}
+
+.image {
+ border: 1px solid #000000 !important;
 }

docs/modules/ROOT/nav-how-to-guides.adoc

@@ -15,6 +15,8 @@
 ** xref:how-to-guides/proc_creating_your_own_environment.adoc[Creating your own environment]
 ** xref:how-to-guides/proc_managing-compliance-with-the-enterprise-contract.adoc[Managing compliance with the Enterprise Contract]
 ** xref:how-to-guides/proc_delete_application.adoc[Deleting an application].
+** xref:how-to-guides/proc_hermetic-builds.adoc[Enabling hermetic builds]
+** xref:how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc[Prefetching package manager dependencies for hermetic build]
 
 
 ////

docs/modules/ROOT/pages/how-to-guides/proc_hermetic-builds.adoc

@@ -0,0 +1,43 @@
+:_content-type: PROCEDURE
+
+[id="hermetic-builds_{context}"]
+= Enabling hermetic builds
+
+Hermetic builds refer to a controlled build process where all required resources and dependencies are declared and prefetched within a build system. This strict control ensures that the build system has complete visibility and control over the dependencies needed for the build, including the ability to accurately collect, inquire, analyze, and manage all dependencies during the build process. By restricting external dependencies and implementing a centralized management approach, hermetic builds provide a predictable and reliable build environment.
+
+IMPORTANT: If you do not vendor all the required dependencies in a git repository while isolated from the network, your build may fail. 
+
+Hermetic builds also focus on establishing a build pipeline that captures and maintains a precise record of the build’s origin, dependencies, and modifications. This emphasis on provenance allows for better traceability, reproducibility, and auditability of the build process. By having comprehensive visibility into the entire pipeline, hermetic builds enable the identification and resolution of issues, facilitate collaboration, and support long-term maintenance of the builds.
+
+
+.Prerequisites
+
+* You have an upgraded build pipeline.
+
+.Procedure
+
+For each component that you want to build hermetically:
+
+. Navigate to your component’s source code.
+
+. Open the *.tekton* directory and locate the *.yaml* files related to the *pull-request* and *push* processes.
+
+. *Configure the hermetic pipeline*: In both the *.yaml* files, add the following hermetic pipeline parameters.
+
++
+[source,yaml]
+----
+param:
+ - name: hermetic
+ value: “true”
+----
+
++
+NOTE: If you add this parameter in a non-java application, that is a link:https://github.com/redhat-appstudio/build-definitions/blob/main/task/buildah/0.1/buildah.yaml[Buildah] task, the build is automatically isolated from the network. This is because all the necessary dependencies are included in a git repository. However, if you want to restrict external network access during the build process, you can pre-fetch dependencies package manager dependencies for the supported languages.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+[role="_additional-resources"]
+.Additional resources
+* For details on pre-fetch dependencies package manager dependencies, see the xref:how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc[Prefetching package manager dependencies for hermetic build]
+

docs/modules/ROOT/pages/how-to-guides/proc_prefetching-dependencies-to-support-hermetic-build.adoc

@@ -0,0 +1,195 @@
+:_content-type: PROCEDURE
+
+[id="prefetching-dependencies-to-support-hermetic-build_{context}"]
+= Prefetching the package manager dependencies for the Hermetic build
+
+{ProductName} uses Cachi2 to prefetch the package manager dependencies for the <<supported-languages>>. Cachi2 is a wrapper around the package managers that makes it easy to prefetch and cache dependencies. It also helps generate a manifest of all dependencies included in your builds, ensuring transparency and maintainability. 
+
+To ensure proper usage of Cachi2, follow these steps for your project’s builds process:
+
+* *Define dependencies* - Explicitly declare dependencies in a lock file generated by the package manager. Cachi2 only fetches dependencies that are declared in this file.
+
+* *Reproducible builds* - Cachi2 enforces exact version matching for dependencies, including transitive dependencies. This ensures controlled and reproducible build dependencies.
+
+* *Secure builds* - While package managers use lock files to resolve dependencies automatically, it is crucial to verify the checksums of all the dependencies to safeguard against supply chain attacks.
+
+[#supported-languages]
+.Supported languages
+
+[cols="1,1"]
+|===
+|Language
+|Package Manager
+
+|Go
+|`gomod`
+
+|Python
+|`pip`
+|===
+
+== Enabling prefetch builds for `gomod`
+
+.Prerequisites
+* You have an upgraded build pipeline.
+* You have dependencies outside your git repository.
+
+.Procedures
+For each component that you want to build hermetically:
+
+. Navigate to your component's source code.
+. Open the *.tekton* directory and locate the *.yaml* files related to the *pull-request* and *push* processes.
+. *Set up dependency prefetching with Cachi2:* In both the `.yaml` files, add the following prefetch-input pipeline parameter.
+
++
+[source,yaml]
+----
+param:
+ - name: prefetch-input
+ value: <package-manager-lock-file> <1>
+----
+<1> If the package manager lock file is located in the root directory of your project, enter `gomod`. Otherwise, if the package manager lock file is in a subdirectory, provide the path to the lock file in JSON format. For example,
+{"type": "{<package-manager>}", "path": "path-to-the-directory-which-contains-the-lockfile}"}
+
+include::../partials/con_hermetic_verification.adoc[]
+
+== Enabling prefetch builds for `pip`
+Cachi2 supports pip by parsing *requirements.txt* files placed in the root of your repository and then downloading the specified dependencies. These files should be lockfiles, which include all the transitive dependencies. Each transitive dependency in the *requirements.txt* file must be pinned to a specific version.
+
+.Prerequisites
+* You have an upgraded build pipeline.
+* You have dependencies outside your git repository.
+
+.Procedures
+For each component that you want to build hermetically:
+
+. Navigate to your component’s source code.
+. Open the *.tekton* directory and locate the *.yaml* files related to the *pull-request* and *push* processes.
+. *Set up dependency prefetching with Cachi2:* In both the `.yaml` files, add the following prefetch-input pipeline parameter.
+
++
+[source,yaml]
+----
+param:
+ - name: prefetch-input
+ value: <package-manager-lock-file> <1>
+----
+<1> If the package manager lock file is located in the root directory of your project, enter `pip`. Otherwise, if the package manager lock file is in a subdirectory, provide the path to the lock file in the JSON format. For example,
+{"type": "{<package-manager>}", "path": "path-to-the-directory-which-contains-the-lockfile}"}
+
+. Create a file (for example, pyproject.toml) with metadata (name and version) and place it in the root of your repository.
+
++
+[source,metadata]
+----
+[metadata]
+name = "my_package"
+version = "0.1.0"
+----
+
++
+NOTE: Instead of a *pyproject.toml* file, you can also create a *setup.py* or *setup.cfg* file. For information on the metadata of these files, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#project-metadata[Project Metadata].
+
+. To generate a fully resolved requirements.txt file, run
+
++
+[source,command]
+----
+$ pip-compile pyproject.toml --generate-hashes
+----
+
++
+NOTE: The command mentioned assumes that you have defined project dependencies in *pyproject.toml*. However, if you have defined the project dependencies in either the *setup.py* or *setup.cfg*, ensure to update the command accordingly.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+== Enabling prefetch builds for `pip` with source dependencies
+If your project depends on `pip` with source dependencies, you can prefetch builds to improve the performance of the build process. Prefetch builds allow Cachi2 to fetch the build dependencies and provide them to Pip for offline installation in a network-isolated environment.
+
+.Prerequisites
+
+* You have an upgraded build pipeline.
+* You have dependencies outside of your git repository.
+* Download the link:https://raw.githubusercontent.com/containerbuildsystem/cachito/master/bin/pip_find_builddeps.py[`pip_find_builddeps.py`] script directly from GitHub.
++
+NOTE: This script has no runtime dependency other than `pip`.
+
+.Procedures
+For each component that you want to build hermetically:
+
+. Navigate to your component's source code.
+. Open the *.tekton* directory and locate the *.yaml* files related to the *pull-request* and *push* processes.
+. *Set up dependency prefetching with Cachi2:* In both the `.yaml` files, add the following prefetch-input pipeline parameter.
+
++
+[source,yaml]
+----
+param:
+ - name: prefetch-input
+ value: <package-manager-lock-file> <1>
+----
+<1> If the package manager lock file is located in the root directory of your project, enter `pip`. Otherwise, if the package manager lock file is in a subdirectory, provide the path to the lock file in the JSON format. For example,
+{"type": "{<package-manager>}", "path": "path-to-the-directory-which-contains-the-lockfile}"}
+
+. Create a file (for example, pyproject.toml) with metadata (name and version) and place it in the root of your repository.
+
++
+[source,metadata]
+----
+[metadata]
+name = "my_package"
+version = "0.1.0"
+----
+
++
+NOTE: Instead of a *pyproject.toml* file, you can also create a *setup.py* or *setup.cfg* file. For information on the metadata of these files, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#project-metadata[Project Metadata].
+
+. Generate a fully resolved requirements.txt file by using the following command:
+
++
+[source,command]
+----
+$ pip-compile pyproject.toml --generate-hashes
+----
+
++
+NOTE: The command mentioned assumes that you have defined project dependencies in *pyproject.toml*. However, if you have defined the project dependencies in either the *setup.py* or *setup.cfg*, ensure to update the command accordingly.
+
+. In the root of your repository create a file, *requirements-build.in*.
+
+. Copy the build system requirements from the *pyproject.toml* file to the *requirements-build.in* file.
+
++
+*For example:*
++
+image::build-requirements.png[Build requirements, role="image"]
+
+. Run the `pip_find_builddeps.py` script and `pip-compile` the outputs by using the following command:
+
++
+[source,command]
+----
+$ pip_find_builddeps.py requirements.txt \
+--append \
+--only-write-on-update \
+-o requirements-build.in
+----
+
+. Use the `pip-compile` command to convert the *requirements-build.in* file in to the *requirements-build.txt* file by using the following command:
+
++
+[source,command]
+----
+$ pip-compile requirements-build.in --allow-unsafe --generate-hashes
+----
+
+. Add the *requirement-build.txt* file to your project. It does not require any changes to your build process. 
+
++
+NOTE: `pip` automatically installs the build dependencies when needed for explicit installation. The purpose of the *requirement-build.txt* file is to enable Cachi2 to fetch the build dependencies and provide them to pip for offline installation in a network-isolated environment.
+
+include::../partials/con_hermetic_verification.adoc[]
+
+[role="_additional-resources"]
+.Additional resources
+* To troubleshoot any issues you experience when working with Enabling prefetch builds for `pip` or `pip` with source dependencies, see the link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/pip.md#troubleshooting[Troubleshooting] section.

docs/modules/ROOT/pages/partials/con_hermetic_verification.adoc

@@ -0,0 +1,5 @@
+.Verification
+
+. On your application, navigate to the *Activity* > *Pipeline* runs tab.
+
+. For a pipeline run with *Type* as *Build*, you can review the status of your build pipeline.

docs/modules/ROOT/partials/con_hermetic_verification.adoc

@@ -0,0 +1,5 @@
+.Verification
+
+. On your application, navigate to the *Activity* > *Pipeline* runs tab.
+
+. For a pipeline run with *Type* as *Build*, you can review the status of your build pipeline.