Make c9s variant contain c9s content only, no OCP content

This is a first stab at openshift#799, aimed at the c9s variant to start.

In this model, the base (container and disk) images we build in the
pipeline do not contain any OCP-specific details. The compose is made up
purely of RPMs coming out directly from the c9s pungi composes.

Let's go over details of this in bullet form:
1. To emphasize the binding to c9s composes, we change the versioning
   scheme: the version string is now *exactly* the same version as the
   pungi compose from which we've built (well, we do add a `.N` field
   because we want to be able to rebuild multiple times on top of the
   same base pungi compose). It's almost like if our builds are part of
   the c9s pungi composes directly. (And maybe one day they will be...)
   This is implemented using a `versionary` script that queries compose
   info.
2. We no longer include `packages-openshift.yaml`: this has all the OCP
   stuff that we want to do in a layered build instead.
3. We no longer completely rewrite `/etc/os-release`. The host *is*
   image-mode CentOS Stream and e.g. `ID` will now say `centos`.
   However, we do still inject `VARIANT` and `VARIANT_ID` fields to
   note that it's of the CoreOS kind. We should probably actually match
   FCOS here and properly add a CoreOS variant in the `centos-release`
   package.
4. Tests which have to do with the OpenShift layer now have the required
   tag `openshift`. This means that it'll no longer run in the default
   set of kola tests. When building the derived image, we will run just
   those tests using `kola run --tag openshift --oscontainer ...`.

Note that to make this work, OCP itself still needs to actually have
that derived image containing the OCP bits. For now, we will build this
in the pipelines (as a separate artifact that we push to the repos) but
the eventual goal is that we'd split that out of the pipeline and have
it be more like how the rest of OCP is built (using Prow/OSBS/Konflux).

Note also we don't currently build the c9s variant in the pipelines but
this is a long time overdue IMO.

Containerfile.openshift

@@ -0,0 +1,8 @@
+# This container is currently built by `cosa buildextend-layered`, which will
+# always set the base container image to be the one that was just built during
+# base compose. In the future when this is a proper separate container image
+# build (e.g. in OpenShift CI or Konflux), this would be a valid ref to the
+# pushed RHCOS base image.
+FROM overridden
+RUN /run/src/config/scripts/apply-manifest /run/src/config/packages-openshift.yaml && \
+  ostree container commit

manifest-c9s.yaml

@@ -1,9 +1,9 @@
-# Manifest for CentOS Stream CoreOS (SCOS)
+# Manifest for CentOS Stream 9 CoreOS Base
 
 rojig:
   license: MIT
   name: scos
-  summary: OKD 4
+  summary: CentOS Stream 9 CoreOS
 
 variables:
   osversion: "c9s"
@@ -12,56 +12,36 @@ variables:
 # common to RHEL 9 & C9S variants
 include:
   - common.yaml
-  - packages-openshift.yaml
-  # Order *after* packages-openshift.yaml because we want to affect
-  # postprocess scripts in it. Confusingly, rpm-ostree include semantics
-  # means that postprocess scripts in latter includes happen before earlier
-  # ones and it's probably too risky to change that now. See also comment in
-  # scos-os-release.yaml
-  - scos-os-release.yaml
 
 # Starting from here, everything should be specific to SCOS
 
 # CentOS Stream 9 repos + internal repos for now
 repos:
   - baseos
   - appstream
-  # CentOS Extras Common repo for SIG RPM GPG keys
-  - extras-common
-  # CentOS NFV SIG repo for openvswitch
-  - sig-nfv
-  # CentOS Cloud SIG repo for cri-o, cri-tools and conmon-rs
-  - sig-cloud-okd
-  # Include RHCOS 9 repo for oc, hyperkube
-  - rhel-9.2-server-ose-4.16
 
-# We include hours/minutes to avoid version number reuse
-automatic-version-prefix: "416.9.<date:%Y%m%d%H%M>"
-# This ensures we're semver-compatible which OpenShift wants
-automatic-version-suffix: "-"
-# Keep this is sync with the version in postprocess
-mutate-os-release: "4.16"
+# Match the format of c9s compose IDs. This field will be driven in the pipeline
+# anyway to match exactly the same compose ID we're composing with so the value
+# here is purely for developer builds.
+automatic-version-prefix: "9-<date:%Y%m%d>.dev"
+
+mutate-os-release: "9"
+
+# Mark the OS as of the CoreOS variant.
+# XXX: should be part of a centos-release subpackage instead
+postprocess:
+  - |
+     #!/usr/bin/bash
+     set -euo pipefail
+     cat >> /usr/lib/os-release <<EOF
+     VARIANT=CoreOS
+     VARIANT_ID=coreos
+     EOF
+
+     # And put "CoreOS" in NAME and PRETTY_NAME
+     sed -i -e 's/^NAME="\(.*\)"/NAME="\1 CoreOS"/' /usr/lib/os-release
+     . /usr/lib/os-release
+     sed -i -e "s/^PRETTY_NAME=.*/PRETTY_NAME=\"$NAME $VERSION\"/" /usr/lib/os-release
 
-# Packages that are only in SCOS and not in RHCOS or that have special
-# constraints that do not apply to RHCOS
 packages:
- # We include the generic release package and tweak the os-release info in a
- # post-proces script
  - centos-stream-release
- # RPM GPG keys for CentOS SIG repos
- - centos-release-cloud-common
- - centos-release-nfv-common
- - centos-release-virt-common
-
-# Packages pinned to specific repos in SCOS 9
-repo-packages:
-  # We always want the kernel from BaseOS
-  - repo: baseos
-    packages:
-      - kernel
-  - repo: appstream
-    packages:
-      # We want the one shipping in C9S, not the equivalently versioned one in RHAOS
-      - nss-altfiles
-      # Use the new containers/toolbox
-      - toolbox

packages-openshift.yaml

@@ -1,3 +1,13 @@
+repos:
+  # c9s doesn't have all the OCP RPMs yet
+  - rhel-9.2-server-ose-4.16
+  - rhel-9.2-fast-datapath
+  # these duplicate manifest-rhel-9.4.yaml and manifest-c9s.yaml, which
+  # `include` us, but in the layering path we don't go through rpm-ostree and
+  # directly apply this manifest
+  - baseos
+  - appstream
+
 packages:
   # The packages below are required by OpenShift/OKD
   # but are not present in CentOS Stream and RHEL.

scripts/apply-manifest

@@ -0,0 +1,88 @@
+#!/usr/bin/python3 -u
+
+# This is a hacky temporary script to apply an rpm-ostree manifest as part of a
+# derived container build. It's only required because we're in this transitional
+# state where some streams use the old way, and others use layering. Once all
+# streams use layering, we should stop using manifests for tha layered bits. (An
+# obvious question here is whether we should keep extending the `rpm-ostree ex
+# rebuild` stuff to keep using manifests even in a layered build. Though likely
+# similar functionality will live in dnf instead.)
+
+# Note this only supports the subset of the manifest spec actually used in
+# `packages-openshift.yaml`.
+
+import os
+import shutil
+import subprocess
+import sys
+import yaml
+
+
+def runcmd(args):
+    print("Running:", ' '.join(args))
+    subprocess.check_call(args)
+
+
+manifest_file = sys.argv[1]
+manifest_dir = os.path.dirname(manifest_file)
+
+with open(manifest_file) as f:
+    manifest = yaml.safe_load(f)
+
+if len(manifest.get('packages', [])):
+
+    packages = []
+    for pkg in manifest['packages']:
+        packages += pkg.split()
+    rpmostree_install = ['rpm-ostree', 'install', '-y'] + packages
+
+    # XXX: temporary hack for cri-o, which wants to create dirs under /opt
+    # https://github.com/CentOS/centos-bootc/issues/393
+    if 'cri-o' in packages:
+        os.makedirs("/var/opt", exist_ok=True)
+
+    # move the canonical dir out of the way
+    moved_yum_repos_d = False
+    if 'repos' in manifest:
+        if os.path.isdir('/etc/yum.repos.d'):
+            # can't just os.rename here because of classic overlayfs
+            # non-POSIXness will hit EXDEV
+            shutil.copytree("/etc/yum.repos.d", "/etc/yum.repos.d.bak")
+            shutil.rmtree("/etc/yum.repos.d")
+            os.mkdir("/etc/yum.repos.d")
+            moved_yum_repos_d = True
+
+        for dir in [manifest_dir, "/run/src/yumrepos"]:
+            for ent in os.listdir(dir):
+                if ent.endswith(".repo"):
+                    shutil.copy(os.path.join(dir, ent), "/etc/yum.repos.d")
+
+        rpmostree_install += ["--disablerepo=*"]
+        for repo in manifest['repos']:
+            rpmostree_install += [f"--enablerepo={repo}"]
+
+        if os.path.isfile("/etc/yum.repos.d/c9s.repo"):
+            # XXX: brutal hack for CentOS GPG key path because our c9s
+            # definitions have key paths that make sense for Fedora, not CentOS
+            # (note gpgcheck is still 1)
+            runcmd(['sed', '-i', '/gpgkey=/ d', '/etc/yum.repos.d/c9s.repo'])
+            # XXX: SHA1 version of the key isn't supported anymore. This isn't
+            # strictly required but squashes a warning. Should check why c9s is
+            # still shipping this; let it error out if it doesn't exist anymore,
+            # that way we'll know to remove this
+            os.unlink('/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras')
+
+        runcmd(rpmostree_install)
+
+    if moved_yum_repos_d:
+        shutil.rmtree("/etc/yum.repos.d")
+        os.rename("/etc/yum.repos.d.bak", "/etc/yum.repos.d")
+
+if len(manifest.get('postprocess', [])):
+    for i, script in enumerate(manifest['postprocess']):
+        name = f"/tmp/postprocess-script-{i}"
+        with open(name, 'w') as f:
+            f.write(script)
+        os.chmod(name, 0o755)
+        runcmd([name])
+        os.unlink(name)

tests/kola/version/rhel-major-version

@@ -10,6 +10,9 @@ set -xeuo pipefail
 
 variant="$(source /etc/os-release; echo "${ID}")"
 case "${variant}" in
+    "centos")
+        osver="$(source /etc/os-release; echo "${VERSION_ID}")"
+        ;;
     "scos")
         osver="$(source /usr/lib/os-release.stream; echo "${VERSION}")"
         ;;

tests/kola/version/rhel-matches-rhcos-build

@@ -12,6 +12,11 @@ set -xeuo pipefail
 ocp_version="$(source /etc/os-release; echo "${VERSION}")"
 variant="$(source /etc/os-release; echo "${ID}")"
 case "${variant}" in
+    "centos")
+        # We skip this in the base SCOS case; our package set is pure CentOS and
+        # the version string matches the compose ID, so this test is invalid.
+        exit 0
+        ;;
     "scos")
         # on SCOS, this is just "9"
         osver="$(source /usr/lib/os-release.stream; echo "${VERSION_ID}")"