Skip to content

Commit

Permalink
Merge pull request #195 from slimreaper35/devel
Browse files Browse the repository at this point in the history
prefetching-dependencies update
  • Loading branch information
arewm authored Dec 17, 2024
2 parents ea6953a + b5110ad commit 3fa275f
Showing 1 changed file with 73 additions and 17 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,37 @@ In {ProductName}, you can run a hermetic build by restricting network access to

For every build, Cachi2 generates a software bill of materials (SBOM) where all dependencies are properly declared and pinned to specific versions. Also, Cachi2 ensures that arbitrary code is never executed during the prefetch, meaning, for example, that the build doesn’t pull any undeclared dependencies. Such measures result in very accurate SBOMs and improve the build reproducibility. For more information about SBOMs, see xref:/how-tos/metadata/sboms.adoc[Inspecting SBOMs].

[#available-languages]
.Available languages
[#available-package-managers]
.Available package managers

[cols="1,1"]
|===
|**Language**
|**Package manager**
|**Programming language**

|xref:gomod[Go]
|`gomod`
|xref:gomod[gomod]
|`Go`

|xref:pip[Python]
|`pip`
|xref:pip[pip]
|`Python`

|xref:npm[Node.js]
|`npm`
|xref:npm[npm]
|`JavaScript`

|xref:rpm[RPM*]
|`rpm`
|xref:yarn[yarn]
|`JavaScript`

|xref:generic[Generic dependencies]
|`generic fetcher`
|===
|xref:bundler[bundler]
|`Ruby`

|xref:rpm*[rpm]
|`N/A`

NOTE: To use Yarn as the package manager, see the link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/usage.md#example-yarn[Cachi2 Usage guide].
|xref:generic[generic]
|`N/A`
|===

NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2/blob/8a870c9cd3b7927e1a2d974b3874140e935aa17e/cachi2/core/resolver.py#L22-L27[RPM package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833].
NOTE: *The link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] and the link:https://github.com/containerbuildsystem/cachi2?tab=readme-ov-file#package-managers[rpm package manager for cachi2] are not fully supported. You can use them to prefetch rpms for your hermetic builds, but the file format and technology may change in the future. If you're interested in the future of this topic, join the discussion at link:https://github.com/rpm-software-management/dnf5/issues/833[rpm-software-management/dnf5#833].
== [[gomod]]Enabling prefetch builds for `gomod`
Expand Down Expand Up @@ -298,6 +302,58 @@ spec:
* From the {ProductName} *Applications* view, go to *Activity > Latest commits*.
//include::../partials/con_hermetic_verification.adoc[]

== [[yarn]]Enabling prefetch builds for `yarn`

Supported versions: 1.x and 3.x. Cachi2 automatically detects the version of `yarn` and fetches any dependencies you declare in your `package.json` and `yarn.lock` project files.

.Prerequisites

* You have an up-to-date `yarn.lock` file in your source repository. To ensure you have the latest `yarn.lock` file or to create it, run the `yarn install` command. If `yarn.lock` is not up-to-date, Cachi2 will not fetch the dependencies.

.Procedure

To prefetch dependencies for a component build, complete the following steps:

. Go to the `.tekton` directory and find the `.yaml` files related to the `*pull request*` and `*push*` processes.
. Configure the hermetic pipeline by adding the following parameters in both `.yaml` files:

+
[source,yaml]
----
spec:
params:
- ...
- name: prefetch-input
value: '{"type": "yarn", "path": "."}'
----

== [[bundler]]Enabling prefetch builds for `bundler`

.Prerequisites

You have a `Gemfile` and a `Gemfile.lock` file in your repository that lists all the dependencies. The `Gemfile.lock` is generated from the `Gemfile` and contains all transitive dependencies pinned to specific versions.

.Procedure

To prefetch dependencies for a component build, complete the following steps:

. Go to the `.tekton` directory and find the `.yaml` files related to the `*pull request*` and `*push*` processes.
. Configure the hermetic pipeline by adding the following parameters in both `.yaml` files:

+
[source,yaml]
----
spec:
params:
- ...
- name: prefetch-input
value: '{"type": "bundler", "path": ".", "allow_binary": "false"}' <1>
----
<1> The `*allow_binary*` parameter specifies whether to allow gems for specific platforms. By default, the value is `false`. If you want to allow gems for specific platforms, set the value to `true`.

. Create a pull request by committing your changes to the repository of the component.
. Review and merge the pull request.

== [[rpm]]Enabling prefetch builds for `rpm`

Cachi2 has a dev-preview package manager capable of fetching `rpm` dependencies. This requires the use of a pair of `rpms.in.yaml` and `rpms.lock.yaml` files to be committed to your repository. You write a `rpms.in.yaml` file and the link:https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#what-is-this[rpm-lockfile-prototype] CLI tool resolves that to produce a `rpms.lock.yaml` file. Cachi2 fetches those specific rpms and enables your build to install them without network access.
Expand Down Expand Up @@ -391,7 +447,7 @@ In {ProductName}, from the *Applications* view, select the application build you

NOTE: Konflux also supports prefetching RPM content which requires a Red Hat subscription. For more information see xref:./activation-keys-subscription.adoc#hermetic-network-isolated-builds[Using Red Hat activation keys to access subscription content].

== [[generic]]Enabling prefetch builds for `Generic dependencies`
== [[generic]]Enabling prefetch builds for `generic fetcher`
If you need to prefetch arbitrary files for your build, Cachi2 supports `generic fetcher` for that purpose. It uses a custom lockfile named `artifacts.lock.yaml` to achieve this. This file needs to be either commited in the source repository, or explicitly specified as an absolute path. The latter is useful in case you for some reason need the lockfile to be dynamic and committing it to the repository would be problematic. For more information on supported types of artifacts, see link:https://github.com/containerbuildsystem/cachi2/blob/main/docs/generic.md[Cachi2 documentation].

.Procedure
Expand Down

0 comments on commit 3fa275f

Please sign in to comment.