Make c9s variant contain c9s content only, no OCP content

This is a first stab at openshift#799, aimed at the c9s variant to start.

In this model, the base (container and disk) images we build in the
pipeline do not contain any OCP-specific details. The compose is made up
purely of RPMs coming out directly from the c9s pungi composes.

Let's go over details of this in bullet form:
1. To emphasize the binding to c9s composes, we change the versioning
   scheme: the version string is now *exactly* the same version as the
   pungi compose from which we've built (well, we do add a `.N` field
   because we want to be able to rebuild multiple times on top of the
   same base pungi compose). It's almost like if our builds are part of
   the c9s pungi composes directly. (And maybe one day they will be...)
   This is implemented using a `versionary` script that queries compose
   info.
2. We no longer include `packages-openshift.yaml`: this has all the OCP
   stuff that we want to do in a layered build instead.
3. We no longer completely rewrite `/etc/os-release`. The host *is*
   image-mode CentOS Stream and e.g. `ID` will now say `centos`.
   However, we do still inject `VARIANT` and `VARIANT_ID` fields to
   note that it's of the CoreOS kind. We should probably actually match
   FCOS here and properly add a CoreOS variant in the `centos-release`
   package.
4. Tests which have to do with the OpenShift layer now have the required
   tag `openshift`. This means that it'll no longer run in the default
   set of kola tests. When building the derived image, we will run just
   those tests using `kola run --tag openshift --oscontainer ...`.

Note that to make this work, OCP itself still needs to actually have
that derived image containing the OCP bits. For now, we will build this
in the pipelines (as a separate artifact that we push to the repos) but
the eventual goal is that we'd split that out of the pipeline and have
it be more like how the rest of OCP is built (using Prow/OSBS/Konflux).

Note also we don't currently build the c9s variant in the pipelines but
this is a long time overdue IMO.

Containerfile.openshift

@@ -0,0 +1,33 @@
+# This builds the final OCP node image on top of the base RHCOS image. The
+# latter may be RHEL or CentOS Stream-based. This is currently only buildable
+# using podman/buildah as it uses some mounting options only available there.
+#
+# To build this, you will want to pass `--security-opt=label=disable` to avoid
+# having to relabel the context directory. Any repos found in `/run/yum.repos.d`
+# will be imported into `/etc/yum.repos.d/` and then removed in the same step (so
+# as to not end up in the final image).
+#
+# Use `--from` to override the base RHCOS image. E.g.:
+#
+# podman build --from quay.io/openshift-release-dev/ocp-v4.0-art-dev:rhel-coreos-base-9.4 ...
+#
+# Or to use a locally built OCI archive:
+#
+# podman build --from oci-archive:builds/latest/x86_64/scos-9-20240416.dev.0-ostree.x86_64.ociarchive ...
+
+# If consuming from repos hosted within the RH network, you'll want to mount in
+# certs too:
+#
+# podman build -v /etc/pki/ca-trust:/etc/pki-ca-trust:ro ...
+#
+# Example invocation:
+#
+# podman build --from oci-archive:$(ls builds/latest/x86_64/*.ociarchive) \
+#   -v rhel-9.4.repo:/run/yum.repos.d/rhel-9.4.repo:ro \
+#   -v /etc/pki/ca-trust:/etc/pki/ca-trust:ro \
+#   --security-opt label=disable -t localhost/openshift-node-c9s \
+#   -f src/config/Containerfile.openshift
+
+FROM quay.io/openshift-release-dev/ocp-v4.0-art-dev:rhel-coreos-base-c9s
+RUN --mount=type=bind,target=/run/src /run/src/scripts/apply-manifest /run/src/packages-openshift.yaml && \
+  ostree container commit

manifest-c9s.yaml

@@ -1,9 +1,9 @@
-# Manifest for CentOS Stream CoreOS (SCOS)
+# Manifest for CentOS Stream 9 CoreOS Base
 
 rojig:
   license: MIT
   name: scos
-  summary: OKD 4
+  summary: CentOS Stream 9 CoreOS
 
 variables:
   osversion: "c9s"
@@ -12,56 +12,36 @@ variables:
 # common to RHEL 9 & C9S variants
 include:
   - common.yaml
-  - packages-openshift.yaml
-  # Order *after* packages-openshift.yaml because we want to affect
-  # postprocess scripts in it. Confusingly, rpm-ostree include semantics
-  # means that postprocess scripts in latter includes happen before earlier
-  # ones and it's probably too risky to change that now. See also comment in
-  # scos-os-release.yaml
-  - scos-os-release.yaml
 
 # Starting from here, everything should be specific to SCOS
 
 # CentOS Stream 9 repos + internal repos for now
 repos:
   - baseos
   - appstream
-  # CentOS Extras Common repo for SIG RPM GPG keys
-  - extras-common
-  # CentOS NFV SIG repo for openvswitch
-  - sig-nfv
-  # CentOS Cloud SIG repo for cri-o, cri-tools and conmon-rs
-  - sig-cloud-okd
-  # Include RHCOS 9 repo for oc, hyperkube
-  - rhel-9.2-server-ose-4.16
 
-# We include hours/minutes to avoid version number reuse
-automatic-version-prefix: "416.9.<date:%Y%m%d%H%M>"
-# This ensures we're semver-compatible which OpenShift wants
-automatic-version-suffix: "-"
-# Keep this is sync with the version in postprocess
-mutate-os-release: "4.16"
+# Match the format of c9s compose IDs. This field will be driven in the pipeline
+# anyway to match exactly the same compose ID we're composing with so the value
+# here is purely for developer builds.
+automatic-version-prefix: "9-<date:%Y%m%d>.dev"
+
+mutate-os-release: "9"
+
+# Mark the OS as of the CoreOS variant.
+# XXX: should be part of a centos-release subpackage instead
+postprocess:
+  - |
+     #!/usr/bin/bash
+     set -euo pipefail
+     cat >> /usr/lib/os-release <<EOF
+     VARIANT=CoreOS
+     VARIANT_ID=coreos
+     EOF
+
+     # And put "CoreOS" in NAME and PRETTY_NAME
+     sed -i -e 's/^NAME="\(.*\)"/NAME="\1 CoreOS"/' /usr/lib/os-release
+     . /usr/lib/os-release
+     sed -i -e "s/^PRETTY_NAME=.*/PRETTY_NAME=\"$NAME $VERSION\"/" /usr/lib/os-release
 
-# Packages that are only in SCOS and not in RHCOS or that have special
-# constraints that do not apply to RHCOS
 packages:
- # We include the generic release package and tweak the os-release info in a
- # post-proces script
  - centos-stream-release
- # RPM GPG keys for CentOS SIG repos
- - centos-release-cloud-common
- - centos-release-nfv-common
- - centos-release-virt-common
-
-# Packages pinned to specific repos in SCOS 9
-repo-packages:
-  # We always want the kernel from BaseOS
-  - repo: baseos
-    packages:
-      - kernel
-  - repo: appstream
-    packages:
-      # We want the one shipping in C9S, not the equivalently versioned one in RHAOS
-      - nss-altfiles
-      # Use the new containers/toolbox
-      - toolbox

scripts/apply-manifest

@@ -0,0 +1,75 @@
+#!/usr/bin/python3 -u
+
+# This is a hacky temporary script to apply an rpm-ostree manifest as part of a
+# derived container build. It's only required because we're in this transitional
+# state where some streams use the old way, and others use layering. Once all
+# streams use layering, we should stop using manifests for tha layered bits. (An
+# obvious question here is whether we should keep extending the `rpm-ostree ex
+# rebuild` stuff to keep using manifests even in a layered build. Though likely
+# similar functionality will live in dnf instead.)
+
+# Note this only supports the subset of the manifest spec actually used in
+# `packages-openshift.yaml`.
+
+import os
+import shutil
+import subprocess
+import sys
+import yaml
+
+
+def runcmd(args):
+    print("Running:", ' '.join(args))
+    subprocess.check_call(args)
+
+
+manifest_file = sys.argv[1]
+manifest_dir = os.path.dirname(manifest_file)
+
+with open(manifest_file) as f:
+    manifest = yaml.safe_load(f)
+
+if len(manifest.get('packages', [])):
+
+    packages = []
+    for pkg in manifest['packages']:
+        packages += pkg.split()
+    rpmostree_install = ['rpm-ostree', 'install', '-y'] + packages
+
+    # XXX: temporary hack for cri-o, which wants to create dirs under /opt
+    # https://github.com/CentOS/centos-bootc/issues/393
+    if 'cri-o' in packages:
+        os.makedirs("/var/opt", exist_ok=True)
+
+    # inject mounted-in repo files
+    extra_repos_dir = '/run/yum.repos.d'
+    copied_repo_files = []
+    if os.path.isdir(extra_repos_dir):
+        for file in os.listdir(extra_repos_dir):
+            src_path = os.path.join(extra_repos_dir, file)
+            if not os.path.isfile(src_path):
+                continue
+            if not file.endswith(".repo"):
+                continue
+            dest_path = os.path.join('/etc/yum.repos.d', file)
+            if os.path.exists(dest_path):
+                raise Exception(f"Repo file {dest_path} already exists")
+            print(f"Copying repo file {file} to /etc/yum.repos.d/")
+            shutil.copy(src_path, dest_path)
+            copied_repo_files += [dest_path]
+
+    runcmd(rpmostree_install)
+
+    # delete the repo files we injected
+    for repo in copied_repo_files:
+        os.unlink(repo)
+
+
+if len(manifest.get('postprocess', [])):
+    for i, script in enumerate(manifest['postprocess']):
+        name = f"/tmp/postprocess-script-{i}"
+        with open(name, 'w') as f:
+            f.write(script)
+        os.chmod(name, 0o755)
+        runcmd([name])
+        os.unlink(name)

tests/kola/version/rhel-major-version

@@ -10,6 +10,9 @@ set -xeuo pipefail
 
 variant="$(source /etc/os-release; echo "${ID}")"
 case "${variant}" in
+    "centos")
+        osver="$(source /etc/os-release; echo "${VERSION_ID}")"
+        ;;
     "scos")
         osver="$(source /usr/lib/os-release.stream; echo "${VERSION}")"
         ;;

tests/kola/version/rhel-matches-rhcos-build

@@ -12,6 +12,11 @@ set -xeuo pipefail
 ocp_version="$(source /etc/os-release; echo "${VERSION}")"
 variant="$(source /etc/os-release; echo "${ID}")"
 case "${variant}" in
+    "centos")
+        # We skip this in the base SCOS case; our package set is pure CentOS and
+        # the version string matches the compose ID, so this test is invalid.
+        exit 0
+        ;;
     "scos")
         # on SCOS, this is just "9"
         osver="$(source /usr/lib/os-release.stream; echo "${VERSION_ID}")"