From 4bffee825ec151e9c009c722570e9517a3b12df8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20=C5=A0oltis?= Date: Thu, 12 Dec 2024 14:32:24 +0100 Subject: [PATCH 1/4] prefetching-dependencies: Update available package managers table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - swap the package manager columns with the language column - use correct programming languages (if applicable) Signed-off-by: Michal Šoltis --- .../configuring/prefetching-dependencies.adoc | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc index 5dcf00ba..5d2d3b12 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc @@ -4,33 +4,33 @@ In {ProductName}, you can run a hermetic build by restricting network access to For every build, Cachi2 generates a software bill of materials (SBOM) where all dependencies are properly declared and pinned to specific versions. Also, Cachi2 ensures that arbitrary code is never executed during the prefetch, meaning, for example, that the build doesn’t pull any undeclared dependencies. Such measures result in very accurate SBOMs and improve the build reproducibility. For more information about SBOMs, see xref:/how-tos/metadata/sboms.adoc[Inspecting SBOMs]. -[#available-languages] -.Available languages +[#available-package-managers] +.Available package managers [cols="1,1"] |=== -|**Language** |**Package manager** +|**Programming language** -|xref:gomod[Go] -|`gomod` +|xref:gomod[gomod] +|`Go` -|xref:pip[Python] -|`pip` +|xref:pip[pip] +|`Python` -|xref:npm[Node.js] -|`npm` +|xref:npm[npm] +|`JavaScript` -|xref:rpm[RPM*] -|`rpm` +|xref:rpm*[rpm] +|`N/A` -|xref:generic[Generic dependencies] -|`generic fetcher` +|xref:generic[generic] +|`N/A` |=== NOTE: To use Yarn as the package manager, see the link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/usage.md#example-yarn[Cachi2 Usage guide]. -NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2/blob/8a870c9cd3b7927e1a2d974b3874140e935aa17e/cachi2/core/resolver.py#L22-L27[RPM package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833]. +NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2/blob/8a870c9cd3b7927e1a2d974b3874140e935aa17e/cachi2/core/resolver.py#L22-L27[rpm package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833]. == [[gomod]]Enabling prefetch builds for `gomod` @@ -391,7 +391,7 @@ In {ProductName}, from the *Applications* view, select the application build you NOTE: Konflux also supports prefetching RPM content which requires a Red Hat subscription. For more information see xref:./activation-keys-subscription.adoc#hermetic-network-isolated-builds[Using Red Hat activation keys to access subscription content]. -== [[generic]]Enabling prefetch builds for `Generic dependencies` +== [[generic]]Enabling prefetch builds for `generic fetcher` If you need to prefetch arbitrary files for your build, Cachi2 supports `generic fetcher` for that purpose. It uses a custom lockfile named `artifacts.lock.yaml` to achieve this. This file needs to be either commited in the source repository, or explicitly specified as an absolute path. The latter is useful in case you for some reason need the lockfile to be dynamic and committing it to the repository would be problematic. For more information on supported types of artifacts, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/generic.md[Cachi2 documentation]. .Procedure From 8026f6bba6fe1c169817c26be22b69e5255248d4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20=C5=A0oltis?= Date: Thu, 12 Dec 2024 14:33:08 +0100 Subject: [PATCH 2/4] prefetching-dependencies: Fix link for (un)supported package managers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michal Šoltis --- .../pages/how-tos/configuring/prefetching-dependencies.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc index 5d2d3b12..41ab762c 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc @@ -30,7 +30,7 @@ For every build, Cachi2 generates a software bill of materials (SBOM) where all NOTE: To use Yarn as the package manager, see the link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/usage.md#example-yarn[Cachi2 Usage guide]. -NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2/blob/8a870c9cd3b7927e1a2d974b3874140e935aa17e/cachi2/core/resolver.py#L22-L27[rpm package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833]. +NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2?tab=readme-ov-file#package-managers[rpm package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833]. == [[gomod]]Enabling prefetch builds for `gomod` From 62e0d0b71aa90dbae2c3fde342d7e0c39c97aacc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20=C5=A0oltis?= Date: Thu, 12 Dec 2024 14:49:43 +0100 Subject: [PATCH 3/4] prefetching-dependencies: Add bundler (Ruby) to supported package managers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michal Šoltis --- .../configuring/prefetching-dependencies.adoc | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc index 41ab762c..2cd4d7c7 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc @@ -21,6 +21,9 @@ For every build, Cachi2 generates a software bill of materials (SBOM) where all |xref:npm[npm] |`JavaScript` +|xref:bundler[bundler] +|`Ruby` + |xref:rpm*[rpm] |`N/A` @@ -298,6 +301,33 @@ spec: * From the {ProductName} *Applications* view, go to *Activity > Latest commits*. //include::../partials/con_hermetic_verification.adoc[] +== [[bundler]]Enabling prefetch builds for `bundler` + +.Prerequisites + +You have a `Gemfile` and a `Gemfile.lock` file in your repository that lists all the dependencies. The `Gemfile.lock` is generated from the `Gemfile` and contains all transitive dependencies pinned to specific versions. + +.Procedure + +To prefetch dependencies for a component build, complete the following steps: + +. Go to the `.tekton` directory and find the `.yaml` files related to the `*pull request*` and `*push*` processes. +. Configure the hermetic pipeline by adding the following parameters in both `.yaml` files: + ++ +[source,yaml] +---- +spec: + params: + - ... + - name: prefetch-input + value: '{"type": "bundler", "path": ".", "allow_binary": "false"}' <1> +---- +<1> The `*allow_binary*` parameter specifies whether to allow gems for specific platforms. By default, the value is `false`. If you want to allow gems for specific platforms, set the value to `true`. + +. Create a pull request by committing your changes to the repository of the component. +. Review and merge the pull request. + == [[rpm]]Enabling prefetch builds for `rpm` Cachi2 has a dev-preview package manager capable of fetching `rpm` dependencies. This requires the use of a pair of `rpms.in.yaml` and `rpms.lock.yaml` files to be committed to your repository. You write a `rpms.in.yaml` file and the link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] CLI tool resolves that to produce a `rpms.lock.yaml` file. Cachi2 fetches those specific rpms and enables your build to install them without network access. From b5110ad3bbf8d5fcbf3f62e3cbeb1d7dd9780176 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20=C5=A0oltis?= Date: Tue, 17 Dec 2024 09:45:37 +0100 Subject: [PATCH 4/4] prefetching-dependencies: Add yarn (JavaScript) to supported package managers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michal Šoltis --- .../configuring/prefetching-dependencies.adoc | 30 +++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc index 2cd4d7c7..3226f206 100644 --- a/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc +++ b/docs/modules/ROOT/pages/how-tos/configuring/prefetching-dependencies.adoc @@ -21,6 +21,9 @@ For every build, Cachi2 generates a software bill of materials (SBOM) where all |xref:npm[npm] |`JavaScript` +|xref:yarn[yarn] +|`JavaScript` + |xref:bundler[bundler] |`Ruby` @@ -31,8 +34,6 @@ For every build, Cachi2 generates a software bill of materials (SBOM) where all |`N/A` |=== -NOTE: To use Yarn as the package manager, see the link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/usage.md#example-yarn[Cachi2 Usage guide]. - NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2?tab=readme-ov-file#package-managers[rpm package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833]. == [[gomod]]Enabling prefetch builds for `gomod` @@ -301,6 +302,31 @@ spec: * From the {ProductName} *Applications* view, go to *Activity > Latest commits*. //include::../partials/con_hermetic_verification.adoc[] +== [[yarn]]Enabling prefetch builds for `yarn` + +Supported versions: 1.x and 3.x. Cachi2 automatically detects the version of `yarn` and fetches any dependencies you declare in your `package.json` and `yarn.lock` project files. + +.Prerequisites + +* You have an up-to-date `yarn.lock` file in your source repository. To ensure you have the latest `yarn.lock` file or to create it, run the `yarn install` command. If `yarn.lock` is not up-to-date, Cachi2 will not fetch the dependencies. + +.Procedure + +To prefetch dependencies for a component build, complete the following steps: + +. Go to the `.tekton` directory and find the `.yaml` files related to the `*pull request*` and `*push*` processes. +. Configure the hermetic pipeline by adding the following parameters in both `.yaml` files: + ++ +[source,yaml] +---- +spec: + params: + - ... + - name: prefetch-input + value: '{"type": "yarn", "path": "."}' +---- + == [[bundler]]Enabling prefetch builds for `bundler` .Prerequisites