Enabling ocp platform mirroring using oc-mirror (#228)

This PR enables platform mirroring using oc mirror plugin by default and gives the option of doing the legacy way as its possible to choose the image org in old way part of #143 --------- Signed-off-by: Mohammed Zeeshan Ahmed <[email protected]> Signed-off-by: Mohammed Zeeshan Ahmed <[email protected]> Co-authored-by: Mohammed Zeeshan Ahmed <[email protected]>
IBM · Dec 7, 2023 · 31b560c · 31b560c
1 parent 31f37d0
commit 31b560c
Show file tree

Hide file tree

Showing 5 changed files with 47 additions and 18 deletions.
diff --git a/docs/run-the-playbooks-for-disconnected.md b/docs/run-the-playbooks-for-disconnected.md
@@ -1,5 +1,7 @@
 # Run the Playbooks
+
 ## Overview
+
 For installing disconnected clusters, you will mostly be following rhe same process as a standard connected cluster.
 
 The main additional steps we would be doing is mirroring the OCP images to another registry which is accessible to
@@ -9,8 +11,10 @@ policy and catalog source, generated by `oc-mirror`, to the cluster.
 Disconnected playbook are mentioned below. Please refer the **4 Run the Playbooks** documentation for details of rest of the playbooks:
 
 * disconnected_mirror_artifacts.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/disconnected_mirror_artifacts.yaml)) - Run before **6_create_nodes.yaml**
-* disconnected_apply_operator_manifests.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/disconnected_apply_operator_manifests.yaml)) - Run after **7_ocp_verification.yaml**
+* disconnected_apply_operator_manifests.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/disconnected_apply_operator_manifests.yaml)) - Run after **7_ocp_verification.yaml**.
+
 ## Pre-requisites
+
 * A running registry where the OCP and operator hub images will be mirrored. If the CA of this registry is not automatically trusted, then keep the 
   CA cert content handy to update in inventory file. The CA cert is the file with which, do dont need to skip tls to access the registry.
 * Make sure you have required pull secrets handy. You will need 2 pull secrets, one to apply on the cluster and another which will be used for 
@@ -28,7 +32,9 @@ Disconnected playbook are mentioned below. Please refer the **4 Run the Playbook
   This registries being mirrored from would typically be the Red Hat registries (registry.redhat.io, quay.io etc)
 * The file server, configured mentioned below.
 * Appropriately updated variables in your `all.yaml`. Refer the variables documentation.
+
 ### File Server
+
 This configuration will take place on the file server mentioned under **File Server** section in overall pre-requisites documentaion. The additional
 configurations are mentioned over here.
 
@@ -47,34 +53,47 @@ configurations are mentioned over here.
       ```
 
   Make sure this directory contains a pre-downloaded `oc-mirror` binary in `tar.gz` format. Currently the supported binary is available for `x86_64` on Red Hat Customer portal openshift [downloads](https://console.redhat.com/openshift/downloads) page. It can also be found on mirror.openshift.com from `4.14` onwards for other architectures.
-### NOTE 
+
+### NOTE
+
 * At this stage, only oc-mirror binary is fetched from File Server, so it is expected that the lpar for disconnected cluster can at least reach `mirror.openshift.com` to download the
   other artifacts for cluster installation.
-* The platorm related image content source policy will be baked into the install config as part of **5 Setup Bastion Playbook**. 
-* Right now on legacy platform mirroring is supported in this playook during the creation of the cluster. 
-* Manifests generated by `oc-mirror` will be applied to the cluster cluster up. So if you add platform details in image set, it
-  will be applied on cluster only after the cluster is up.
+* The platform related image content source policy will be baked into the install config as part of **5 Setup Bastion Playbook**.
+* For platform content, mirroring is supported both using `oc-mirror` plugin as well as legacy way. 
+* `oc-mirror` is used as default alhough it is possible to switch to using the legacy way of mirroing platform seperately as well. **NOTE**: Only legacy way supports specifying your own org on the registry for the ocp images.
+* Manifests generated by `oc-mirror` will be applied to the cluster once it is up.
 
 ## Disconnected Mirror Artifacts Playbook
+
 ### Overview
+
 Mirror the ocp platform and other necessary images to the mirror registry. Please run this playbook before you run **6 Create Nodes Playbook** and after
 **0 Setup Playbook**.
+
 ### Outcomes
+
 * Download `oc` and `oc-mirror` to the mirror host.
 * Template the mirror pull secret to the mirror host.
 * Add the ca cert to the mirror host anchors if ca is not trusted.
 * Mirror the platform images using `oc adm release mirror` if legacy mirroring is enabled.
 * Template the image set to mirror host and then mirror it using `oc-mirror` plogin.
 * Copy the results on the `oc-mirror` to ansible controller to apply to cluster in future steps.
+
 ### Notes
-* Currently, platform can **only** be mirrored the legacy way. While the image set can contain platform mirroring configs, it will **not** be applied to cluster during creation. 
-* This playbook can be run at any stage after the **0 Setup** playbook. Make sure to run this before the cluster starts pulling at the images from the registry 
+
+* Platform can be mirrored both using `oc-mirror` as well as legacy way, using `oc adm catalog mirror`.
+* `oc-mirror` is default method but you can also use legacy mirroring. `oc-mirror` manifests will be only be applied on the cluster, post verification using below playbook.
+* This playbook can be run at any stage after the **0 Setup** playbook. Make sure to run this before the cluster starts pulling at the images from the registry
   which typically happens where the **Create Nodes Playbook** is run.
 
 # Disconnected apply oc mirror manifests to cluster Playbook
+
 ### Overview
+
 Post cluster creation, `oc-mirror` manifests are applied to the cluster. Please run this playbook after **7 OCP Verification Playbook**.
+
 ### Outcomes
+
 * Copy the `oc-mirror` results manifests to the bastion.
 * Apply the copied manifests to the cluster.
 * Disable default content sources.
diff --git a/docs/set-variables-group-vars.md b/docs/set-variables-group-vars.md
@@ -257,19 +257,20 @@
 **disconnected.mirroring.host.pass** | String containing the password of the host, which will be used for mirroring | mirrorpassword
 **disconnected.mirroring.file_server.clients_dir** | Directory path relative to the HTTP/FTP accessible directory on **env.file_server**<br /> where client binary tarballs are kept | clients
 **disconnected.mirroring.file_server.oc_mirror_tgz** | Name of oc-mirror tarball on **env.file_server** in **disconnected.mirroring.file_server.clients_dir** | oc-mirror.tar.gz
-**disconnected.mirroring.legacy.platform** | True or False if the platform should be mirrored using `oc adm release mirror`. | True
+**disconnected.mirroring.legacy.platform** | True or False if the platform should be mirrored using `oc adm release mirror`. | False
 **disconnected.mirroring.legacy.ocp_quay_release_image_tag** | The tag of the release image *quay.io/openshift-release-dev/ocp-release* to mirror and use | 4.13.1-s390x
 **disconnected.mirroring.legacy.ocp_org** | The org part of the repo on the mirror registry where the release image will be pushed | ocp4
 **disconnected.mirroring.legacy.ocp_repo** | The repo part of the repo on the mirror registry where the release image will be pushed | openshift4
 **disconnected.mirroring.legacy.ocp_tag** | The tag part of the repo on the mirror registry where the release image will be pushed.<br /> Full image would be as below.:<br /><br /> disconnected.registry.url/disconnected.mirroring.legacy.ocp_org/disconnected...ocp_repo<br />:disconnected..ocp_tag | v4.13.1
+**disconnected.mirroring.oc_mirror.release_image_tag** | The ocp release image tag you want to install the cluster with. Used when legacy platform <br /> mirroring is disabled and **disconnected.mirroring.oc_mirror.image_set** contains platform <br /> entries. |  4.13.1-multi
 **disconnected.mirroring.oc_mirror.oc_mirror_args.continue_on_error** | True or False to give `--continue-on-error` flag to `oc-mirror` | False
 **disconnected.mirroring.oc_mirror.oc_mirror_args.source_skip_tls** | True or False to give `--source-skip-tls` flag to `oc-mirror` | False
 **disconnected.mirroring.oc_mirror.image_set** | YAML fields containing a standard `oc-mirror` [image set](https://docs.openshift.com/container-platform/latest/installing/disconnected_install/installing-mirroring-disconnected.html#oc-mirror-creating-image-set-config_installing-mirroring-disconnected) with some minor changes to schema. <br /> Differences are documented as needed. Used to generate final image set. | see template
 **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.enabled** | True or False to use registry storage backend for pushing mirrored content directly to the registry. <br /> Currently only this backend is supported.| True
 **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.org** | The org part of registry imageURL from standard image set. | mirror
-**disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.repo** | The repo part of registry imageURL from standard image set. <br /> Final imageURL will be as below:<br /> <br /> disconnected.registry.url/disconnected.mirroring.oc_mirror.image_set.storageConfig<br />.registry.imageURL.org/disconnected...imageURL.repo | oc-mirror-metadata
+**disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.imageURL.repo** | The repo part of registry imageURL from standard image set. <br /> Final imageURL will be as below:<br /> <br /> disconnected.registry.url/disconnected.mirroring.oc_mirror.image_set.storageConfig <br />.registry.imageURL.org/disconnected...imageURL.repo | oc-mirror-metadata
 **disconnected.mirroring.oc_mirror.image_set.storageConfig.registry.skipTLS** | True of False same purpose served as in standard image set i.e. skip the tls for the registry<br />   during mirroring.| false
-**disconnected.mirrroing.oc_mirror.image_set.mirror** | YAML containing a list of what needs to be mirrored. See the oc mirror image set documentation. <br /> *WARNING*: Platform mirroring in this way is not supported. Use legacy way of platform mirroring | see oc-mirror [image set](https://docs.openshift.com/container-platform/latest/installing/disconnected_install/installing-mirroring-disconnected.html#oc-mirror-creating-image-set-config_installing-mirroring-disconnected)   documentation
+**disconnected.mirrroing.oc_mirror.image_set.mirror** | YAML containing a list of what needs to be mirrored. See the oc mirror image set documentation. | see oc-mirror [image set](https://docs.openshift.com/container-platform/latest/installing/disconnected_install/installing-mirroring-disconnected.html#oc-mirror-creating-image-set-config_installing-mirroring-disconnected)   documentation
 
 ## 18 - (Optional) Create compute node in a day-2 operation
 

diff --git a/inventories/default/group_vars/all.yaml.template b/inventories/default/group_vars/all.yaml.template
@@ -332,7 +332,7 @@ disconnected:
       ocp_download_url: "https://mirror.openshift.com/pub/openshift-v4/multi/clients/ocp/4.13.1/amd64/"
       ocp_client_tgz: 'openshift-client-linux.tar.gz' # name of the oc binary. Should be a tar.gz file
     legacy:
-      platform: True # if true then platform is mirrored in the old way and install config will be patched with the imagecontentsourcepolicy
+      platform: False # if true then platform is mirrored in the old way and install config will be patched with the imagecontentsourcepolicy
       ocp_quay_release_image_tag: '4.13.1-s390x'
       ocp_org: 'ocp4'
       ocp_repo: 'openshift4'
@@ -341,6 +341,7 @@ disconnected:
       oc_mirror_args:
         continue_on_error: False
         source_skip_tls: False
+      release_image_tag: '4.13.1-multi'
       image_set: # this field is a standard image set from oc-mirror documentation. The only exception is the storageConfig which is altered to allow substitution of disconnected.registry.url
         storageConfig:
           registry:
@@ -349,7 +350,15 @@ disconnected:
               org: mirror
               repo: oc-mirror-metadata
             skipTLS: false # standard field form oc-mirror schema
-        mirror: # this field is also atandard from the oc-mirror schema. It will be substituted as is into the final image set # WARNING: platform mirroring through oc-mirror is currently not supported, use legacy
+        mirror: # this field is also atandard from the oc-mirror schema. It will be substituted as is into the final image set.
+          platform:
+            architectures:
+            - multi # note: while image tags such as `multi-s390x` are also available on quay, we cannot mirror these, so list can only contain pure architecture names such as `s390x` or the multiarch `multi`
+            channels:
+            - name: stable-4.13
+              full: false
+              minVersion: 4.13.1
+              maxVersion: 4.13.1
           operators:
             - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.13
               full: false

diff --git a/roles/get_ocp/tasks/main.yaml b/roles/get_ocp/tasks/main.yaml
@@ -90,7 +90,7 @@
   tags: get_ocp
   ansible.builtin.shell: |
     set -o pipefail
-    {{ 'export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=quay.io/openshift-release-dev/ocp-release:' + disconnected.mirroring.legacy.ocp_quay_release_image_tag if disconnected.enabled and disconnected.mirroring.legacy.platform }}
+    {{ 'export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=quay.io/openshift-release-dev/ocp-release:' if disconnected.enabled }}{{ disconnected.mirroring.oc_mirror.release_image_tag if disconnected.enabled and not disconnected.mirroring.legacy.platform }}{{ disconnected.mirroring.legacy.ocp_quay_release_image_tag if disconnected.enabled and disconnected.mirroring.legacy.platform }}
     /root/ocpinst/openshift-install create manifests --dir=/root/ocpinst/
   become: true
 
@@ -127,7 +127,7 @@
   become: true
   ansible.builtin.shell: |
     set -o pipefail
-    {{ 'export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=quay.io/openshift-release-dev/ocp-release:' + disconnected.mirroring.legacy.ocp_quay_release_image_tag if disconnected.enabled and disconnected.mirroring.legacy.platform }}
+    {{ 'export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=quay.io/openshift-release-dev/ocp-release:' if disconnected.enabled }}{{ disconnected.mirroring.oc_mirror.release_image_tag if disconnected.enabled and not disconnected.mirroring.legacy.platform }}{{ disconnected.mirroring.legacy.ocp_quay_release_image_tag if disconnected.enabled and disconnected.mirroring.legacy.platform }}
     /root/ocpinst/openshift-install create ignition-configs  --dir=/root/ocpinst/
 
 - name: Set ownership to root and permissions of ignitions and related files.

diff --git a/roles/get_ocp/templates/install-config.yaml.j2 b/roles/get_ocp/templates/install-config.yaml.j2
@@ -40,13 +40,13 @@ platform:
   none: {}
 fips: {{ env.install_config.fips }}
 pullSecret: '{{ env.redhat.pull_secret if not disconnected.enabled else disconnected.registry.pull_secret }}'
-{% if disconnected.enabled and disconnected.mirroring.legacy.platform %}
+{% if disconnected.enabled %}
 {{ 'imageContentSources: ' }}
 {{ '- mirrors:'}}
-{{ '  - ' + disconnected.registry.url + '/' + disconnected.mirroring.legacy.ocp_org + '/' + disconnected.mirroring.legacy.ocp_repo }}
+{{ '  - ' + disconnected.registry.url + '/' }}{{ disconnected.mirroring.legacy.ocp_org if disconnected.mirroring.legacy.platform else 'openshift' }}{{ '/' }}{{ disconnected.mirroring.legacy.ocp_repo if disconnected.mirroring.legacy.platform else 'release-images' }}
 {{ '  source: quay.io/openshift-release-dev/ocp-release' }}
 {{ '- mirrors:'}}
-{{ '  - ' + disconnected.registry.url + '/' + disconnected.mirroring.legacy.ocp_org + '/' + disconnected.mirroring.legacy.ocp_repo }}
+{{ '  - ' + disconnected.registry.url + '/' }}{{ disconnected.mirroring.legacy.ocp_org if disconnected.mirroring.legacy.platform else 'openshift' }}{{ '/' }}{{ disconnected.mirroring.legacy.ocp_repo if disconnected.mirroring.legacy.platform else 'release' }}
 {{ '  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev' }}
 {% endif %}
 {% if disconnected.enabled and not disconnected.registry.ca_trusted %}