Skip to content

Commit

Permalink
Make openshift install and proxy vars default (#327)
Browse files Browse the repository at this point in the history
Tested with sno and 3-node cluster on KVM (NAT).
run playbooks 5,6 and 7.
Test to specify only partial vars for install_config:
install_config:
   control:
       hyperthreading: Enabled
   machine_network: 192.168.122.0/16

---------

Signed-off-by: Amadeuds Podvratnik <[email protected]>
  • Loading branch information
AmadeusPodvratnik authored Sep 11, 2024
1 parent 5c56cfc commit 748bfe3
Show file tree
Hide file tree
Showing 13 changed files with 142 additions and 80 deletions.
58 changes: 32 additions & 26 deletions docs/set-variables-group-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,29 +148,7 @@
**env.cluster.nodes.infra.ipv6** | <b>(Optional)</b> IPv6 address of the infra nodes. iThis list can be expanded to any number of nodes, minimum 2. Use provided list formatting (if use_ipv6 variable is 'True'). | fd00::10fd00::11
**env.cluster.nodes.infra.hostname** | <b>(Optional)</b> Hostnames for infra nodes. Must match the total number of IP addresses for infra nodes. If DNS is hosted on the bastion, this can be anything. If DNS is hosted elsewhere, this must match DNS definition. This will be combined with the metadata_name and base_domain to create a Fully Qualififed Domain Name (FQDN). | infra-01infra-02

## 11 - OpenShift Settings
**Variable Name** | **Description** | **Example**
:--- | :--- | :---
**env.install_config.api_version** | Kubernetes API version for the cluster. These install_config variables will be passed to the OCP install_config file. This file is templated in the get_ocp role during the setup_bastion playbook. To make more fine-tuned adjustments to the install_config, you can find it at roles/get_ocp/templates/install-config.yaml.j2 | v1
**env.install_config.compute.architecture** | Computing architecture for the compute nodes. Must be s390x for clusters on IBM zSystems. | s390x
**env.install_config.compute.hyperthreading** | Enable or disable hyperthreading on compute nodes. Recommended enabled. | Enabled
**env.install_config.control.architecture** | Computing architecture for the control nodes. Must be s390x for clusters on IBM zSystems, amd64 for Intel or AMD systems, and arm64 for ARM servers. | s390x
**env.install_config.control.hyperthreading** | Enable or disable hyperthreading on control nodes. Recommended enabled. | Enabled
**env.install_config.cluster_network.cidr** | IPv4 block in Internal cluster networking in Classless Inter-Domain Routing (CIDR) notation. Recommended to keep as is. | 10.128.0.0/14
**env.install_config.cluster_network.host_prefix** | The subnet prefix length to assign to each individual node. For example, if hostPrefix is set to 23 then each node is assigned a /23 subnet out of the given cidr. A hostPrefix value of 23 provides 510 (2^(32 - 23) - 2) pod IP addresses. | 23
**env.install_config.cluster_network.type** | The cluster network provider Container Network Interface (CNI) plug-in to install. Either OpenShiftSDN or OVNKubernetes (default). | OVNKubernetes
**env.install_config.service_network** | The IP address block for services. The default value is 172.30.0.0/16. The OpenShift SDN and OVN-Kubernetes network providers support only a single IP address block for the service network. An array with an IP address block in CIDR format. | 172.30.0.0/16
**env.install_config.machine_network** | The IP address block for Nodes IP Pool. The default value is 192.168.122.0/24 For NAT Network Mode. In case of MacvTap it will be depend on Inteface IP assignment. An array with an IP address block in CIDR format. | 192.168.122.0/24
**env.install_config.fips** | True or False (boolean) for whether or not to use the United States' Federal Information Processing Standards (FIPS). Not yet certified on IBM zSystems. Enclosed in 'single quotes'. | 'false'

## 12 - (Optional) Proxy
**Variable Name** | **Description** | **Example**
:--- | :--- | :---
**env.proxy.http** | (Optional) A proxy URL to use for creating HTTP connections outside the cluster. Will be used in the install-config and applied to other Ansible hosts unless set otherwise in no_proxy below. Must follow this pattern: http://username:pswd>@ip:port | http://ocp-admin:[email protected]:80
**env.proxy.https** | (Optional) A proxy URL to use for creating HTTPS connections outside the cluster. Will be used in the install-config and applied to other Ansible hosts unless set otherwise in no_proxy below. Must follow this pattern: https://username:pswd@ip:port | https://ocp-admin:[email protected]:80
**env.proxy.no** | (Optional) A comma-separated list (no spaces) of destination domain names, IP addresses, or other network CIDRs to exclude from proxying. When using a proxy, all necessary IPs and domains for your cluster will be added automatically. See roles/get_ocp/templates/install-config.yaml.j2 for more details on the template. Preface a domain with . to match subdomains only. For example, .y.com matches x.y.com, but not y.com. Use * to bypass the proxy for all listed destinations. | example.com,192.168.10.1

## 13 - (Optional) Misc
## 11 - (Optional) Misc
**Variable Name** | **Description** | **Example**
:--- | :--- | :---
**env.language** | What language would you like Red Hat Enterprise Linux to use? In UTF-8 language code. Available languages and their corresponding codes can be found [here](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html-single/international_language_support_guide/index), in the "Locale" column of Table 2.1. | en_US.UTF-8
Expand All @@ -188,7 +166,7 @@
**env.jumphost.pass** | (Optional) The password for user to login to the jumphost. | ch4ngeMe!
**env.jumphost.path_to_keypair** | (Optional) The absolute path to the public key file on the jumphost to be copied to the bastion. | /home/admin/.ssh/id_rsa.pub

## 14 - OCP and RHCOS (CoreOS)
## 12 - OCP and RHCOS (CoreOS)

**Variable Name** | **Description** | **Example**
:--- | :--- | :---
Expand All @@ -201,7 +179,7 @@
**rhcos_live_initrd** | CoreOS initramfs to be used for the bootstrap, control and compute nodes. | rhcos-4.12.3-s390x-live-initramfs.s390x.img
**rhcos_live_rootfs** | CoreOS rootfs to be used for the bootstrap, control and compute nodes. | rhcos-4.12.3-s390x-live-rootfs.s390x.img

## 15 - (Optional) Create compute node in a day-2 operation
## 13 - (Optional) Create compute node in a day-2 operation

**Variable Name** | **Description** | **Example**
:--- | :--- | :---
Expand All @@ -215,14 +193,34 @@
**day2_compute_node.host_user** | KVM host user which is used to create the VM | root
**day2_compute_node.host_arch** | KVM host architecture. | s390x

## 16 - (Optional) Agent Based Installer
## 14 - (Optional) Agent Based Installer
**Variable Name** | **Description** | **Example**
:--- | :--- | :---
**abi.flag** | This is the flag, Will be used to identify during execution. Few checks in the playbook will be depend on this (default value will be False) | True
**abi.ansible_workdir** | This will be work directory name, it will keep required data that need to be present during or after execution | ansible_workdir
**abi.ocp_installer_version** | Version will contain value of openshift-installer binary version user desired to be used | '4.15.0-rc.8'
**abi.ocp_installer_url** | This is the base url of openshift installer binary it will remain same as static value, User Do not need to give value until user wants to change the mirror | 'https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp/'

## OpenShift Settings
* The parameters bellow have a hierachical structure and need to be added to all.yaml in given format. For example if you want to change the hyperthreading (disable) than you need to specify the following value in all.yaml file:
install_config:
compute:
hyperthreading: Disabled

**Variable Name** | **Description** | **Example/Default**
:--- | :--- | :---
**install_config.api_version** | Kubernetes API version for the cluster. These install_config variables will be passed to the OCP install_config file. This file is templated in the get_ocp role during the setup_bastion playbook. To make more fine-tuned adjustments to the install_config, you can find it at roles/get_ocp/templates/install-config.yaml.j2 | v1
**install_config.compute.architecture** | Computing architecture for the compute nodes. Must be s390x for clusters on IBM zSystems. | s390x
**install_config.compute.hyperthreading** | Enable or disable hyperthreading on compute nodes. Recommended enabled. | Enabled
**install_config.control.architecture** | Computing architecture for the control nodes. Must be s390x for clusters on IBM zSystems, amd64 for Intel or AMD systems, and arm64 for ARM servers. | s390x
**install_config.control.hyperthreading** | Enable or disable hyperthreading on control nodes. Recommended enabled. | Enabled
**install_config.cluster_network.cidr** | IPv4 block in Internal cluster networking in Classless Inter-Domain Routing (CIDR) notation. Recommended to keep as is. | 10.128.0.0/14
**install_config.cluster_network.host_prefix** | The subnet prefix length to assign to each individual node. For example, if hostPrefix is set to 23 then each node is assigned a /23 subnet out of the given cidr. A hostPrefix value of 23 provides 510 (2^(32 - 23) - 2) pod IP addresses. | 23
**install_config.cluster_network.type** | The cluster network provider Container Network Interface (CNI) plug-in to install. Either OpenShiftSDN or OVNKubernetes (default). | OVNKubernetes
**install_config.service_network** | The IP address block for services. The default value is 172.30.0.0/16. The OpenShift SDN and OVN-Kubernetes network providers support only a single IP address block for the service network. An array with an IP address block in CIDR format. | 172.30.0.0/16
**install_config.machine_network** | The IP address block for Nodes IP Pool. The default value is 192.168.122.0/24 For NAT Network Mode. In case of MacvTap it will be depend on Inteface IP assignment. An array with an IP address block in CIDR format. | 192.168.122.0/24
**install_config.fips** | True or False (boolean) for whether or not to use the United States' Federal Information Processing Standards (FIPS). Not yet certified on IBM zSystems. Enclosed in 'single quotes'. | 'false'

## Packages (Optional)
* Packages are installed based on the executed playbooks based on the given requirements. This means that these variables have default values which can be overwritten in all.yaml file.
* The following table describe the current installed packages and their default values.
Expand All @@ -237,6 +235,14 @@
**pkgs_bastion** | A list of packages that will be installed on the bastion during the setup_bastion playbook. Feel free to add more as needed, just make sure to follow the same list format. | [ haproxy, httpd, bind, bind-utils, expect, firewalld, mod_ssl, python3-policycoreutils, rsync ]
**pkgs_zvm** | A list of packages that will be installed in case of HCP (zVM nodes) or LPAR installation. | [ git, python3-pip, python3-devel, openssl-devel, rust, cargo, libffi-devel, wget, tar, jq, gcc, make, x3270, python39 ]

## Proxy (Optional)
**Variable Name** | **Description** | **Example**
:--- | :--- | :---
**use_proxy** | (Optional) Use proxyx . Default value is 'False'. Possible values are 'True' or 'False'. | False
**proxy_http** | (Optional) A proxy URL to use for creating HTTP connections outside the cluster. Will be used in the install-config and applied to other Ansible hosts unless set otherwise in no_proxy below. Must follow this pattern: http://username:pswd>@ip:port | http://ocp-admin:[email protected]:80
**proxy_https** | (Optional) A proxy URL to use for creating HTTPS connections outside the cluster. Will be used in the install-config and applied to other Ansible hosts unless set otherwise in no_proxy below. Must follow this pattern: https://username:pswd@ip:port | https://ocp-admin:[email protected]:80
**proxy_no** | (Optional) A comma-separated list (no spaces) of destination domain names, IP addresses, or other network CIDRs to exclude from proxying. When using a proxy, all necessary IPs and domains for your cluster will be added automatically. See roles/get_ocp/templates/install-config.yaml.j2 for more details on the template. Preface a domain with . to match subdomains only. For example, .y.com matches x.y.com, but not y.com. Use * to bypass the proxy for all listed destinations. | example.com,192.168.10.1

## Disconnected cluster setup (Optional)
**Variable Name** | **Description** | **Example**
:--- | :--- | :---
Expand Down
40 changes: 13 additions & 27 deletions inventories/default/group_vars/all.yaml.template
Original file line number Diff line number Diff line change
Expand Up @@ -187,30 +187,7 @@ env:
# All variables below this point do not need to be changed for a default installation #
#######################################################################################

# Section 11 - OpenShift Settings
install_config:
api_version: v1
compute:
architecture: s390x
hyperthreading: Enabled
control:
architecture: s390x
hyperthreading: Enabled
cluster_network:
cidr: 10.128.0.0/14
host_prefix: 23
type: OVNKubernetes
service_network: 172.30.0.0/16
machine_network: 192.168.122.0/24
fips: 'false'

# Section 12 - (Optional) Proxy
# proxy:
# http:
# https:
# no:

# Section 13 - (Optional) Misc
# Section 11 - (Optional) Misc
language: en_US.UTF-8
timezone: America/New_York
keyboard: us
Expand All @@ -231,7 +208,7 @@ env:
pass:
path_to_keypair:

# Section 14 - OCP and RHCOS (CoreOS)
# Section 12 - OCP and RHCOS (CoreOS)

# ocp_download_url with '/' at the end !
ocp_download_url: "https://mirror.openshift.com/pub/openshift-v4/multi/clients/ocp/4.13.1/s390x/"
Expand All @@ -251,7 +228,7 @@ rhcos_live_initrd: "rhcos-4.12.3-s390x-live-initramfs.s390x.img"
rhcos_live_rootfs: "rhcos-4.12.3-s390x-live-rootfs.s390x.img"


# Section 15 - (Optional) Create additional compute node in a day-2 operation
# Section 13 - (Optional) Create additional compute node in a day-2 operation
day2_compute_node:
vm_name:
vm_hostname:
Expand All @@ -264,15 +241,24 @@ day2_compute_node:
host_arch:


# Section 16 - Agent Based Installer ( Optional )
# Section 14 - Agent Based Installer ( Optional )
abi:
flag: False
ansible_workdir: 'ansible_workdir'
ocp_installer_version: '4.15.0-rc.8'
ocp_installer_url: 'https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp/'
boot_method: <pxe|iso>

# Openshift Settings
# Settings like architecture, hyperthreading, network cidr, etc.
# This is a hierachical structure having install_config as top level.
# Check the documentation for parameters and default/example values.

# (Optional) Packages
# Format to overwrite the packages to be installed is:
# pkgs_controller: [ openssh, expect, sshuttle ]
# Overwrite exists for pkgs_galaxy, pkgs_controller, pkgs_kvm, pkgs_bastion, pkgs_zvm

# (Optional) Proxy
# Pls check the documentation which vars are present (include examples). If use_proxy is set to true,
# than proxy_http, proxy_https and proxy_no must be set.
16 changes: 16 additions & 0 deletions roles/create_bastion/defaults/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Openshift Settings
install_config_defaults:
api_version: v1
compute:
architecture: s390x
hyperthreading: Enabled
control:
architecture: s390x
hyperthreading: Enabled
cluster_network:
cidr: 10.128.0.0/14
host_prefix: 23
type: OVNKubernetes
service_network: 172.30.0.0/16
machine_network: 192.168.122.0/24
fips: 'false'
2 changes: 1 addition & 1 deletion roles/create_bastion/templates/bastion-ks.cfg.j2
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ bootloader --append="crashkernel=auto" --location=mbr --boot-drive=vda
clearpart --all --initlabel --drives=vda

# Disk partitioning information
{% if env.install_config.control.architecture == 'arm64' %}
{% if install_config_vars.control.architecture == 'arm64' %}
# TODO: Special setup for arm required, because our arm server requires /boot/efi partition with efi file system
ignoredisk --only-use=vda
# System bootloader configuration
Expand Down
2 changes: 1 addition & 1 deletion roles/create_bastion/templates/rhel9-bastion-ks.cfg.j2
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ bootloader --append="crashkernel=auto" --location=mbr --boot-drive=vda
clearpart --all --initlabel --drives=vda

# Disk partitioning information
{% if env.install_config.control.architecture == 'arm64' %}
{% if install_config.control.architecture == 'arm64' %}
# TODO: Special setup for arm required, because our arm server requires /boot/efi partition with efi file system
ignoredisk --only-use=vda
# System bootloader configuration
Expand Down
6 changes: 6 additions & 0 deletions roles/create_bastion/vars/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
install_config_vars: |
{%- if install_config is defined and install_config is iterable -%}
{{ install_config_defaults | combine (install_config, recursive=True) }}
{%- else -%}
{{ install_config_defaults }}
{%- endif -%}
22 changes: 22 additions & 0 deletions roles/get_ocp/defaults/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Openshift Settings
install_config_defaults:
api_version: v1
compute:
architecture: s390x
hyperthreading: Enabled
control:
architecture: s390x
hyperthreading: Enabled
cluster_network:
cidr: 10.128.0.0/14
host_prefix: 23
type: OVNKubernetes
service_network: 172.30.0.0/16
machine_network: 192.168.122.0/24
fips: 'false'

# (Optional) Proxy
use_proxy: False
proxy_http:
proxy_https:
proxy_no:
2 changes: 0 additions & 2 deletions roles/get_ocp/tasks/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,6 @@

- name: Use template file to create install-config and backup.
tags: get_ocp
vars:
use_proxy: "{{ 'True' if (proxy_env.http_proxy is defined or proxy_env.https_proxy is defined or proxy_env.no_proxy is defined) else 'False' }}"
template:
src: install-config.yaml.j2
dest: "{{ item }}"
Expand Down
26 changes: 13 additions & 13 deletions roles/get_ocp/templates/install-config.yaml.j2
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
apiVersion: {{ env.install_config.api_version }}
apiVersion: {{ install_config_vars.api_version }}
baseDomain: {{ env.cluster.networking.base_domain }}
{% if use_proxy == true %}
{{ 'proxy: ' }}
{{ ' httpProxy: ' + env.proxy.http }}
{{ ' httpsProxy: ' + env.proxy.https }}
{{ ' noProxy: ' + env.proxy.no + ',' + '127.0.0.1,' + 'localhost,' +
{{ ' httpProxy: ' + proxy_http }}
{{ ' httpsProxy: ' + proxy_https }}
{{ ' noProxy: ' + proxy_no + ',' + '127.0.0.1,' + 'localhost,' +
env.bastion.networking.ip + ',' +
env.cluster.nodes.bootstrap.ip + ',' +
env.cluster.nodes.control.ip|join(',') + ',' +
Expand All @@ -18,27 +18,27 @@ env.cluster.nodes.infra.ip|join(',')+ ',' +
'.' + env.cluster.networking.metadata_name + '.' + env.cluster.networking.base_domain }}
{% endif %}
compute:
- hyperthreading: {{ env.install_config.compute.hyperthreading }}
- hyperthreading: {{ install_config_vars.compute.hyperthreading }}
name: worker
replicas: {{ (env.cluster.nodes.compute.hostname | default('') | length) }}
architecture: {{ env.install_config.compute.architecture }}
architecture: {{ install_config_vars.compute.architecture }}
controlPlane:
hyperthreading: {{ env.install_config.control.hyperthreading }}
hyperthreading: {{ install_config_vars.control.hyperthreading }}
name: master
replicas: {{(env.cluster.nodes.control.hostname | length)}}
architecture: {{ env.install_config.control.architecture }}
architecture: {{ install_config_vars.control.architecture }}
metadata:
name: {{ env.cluster.networking.metadata_name }}
networking:
clusterNetwork:
- cidr: {{ env.install_config.cluster_network.cidr }}
hostPrefix: {{ env.install_config.cluster_network.host_prefix }}
networkType: {{ env.install_config.cluster_network.type }}
- cidr: {{ install_config_vars.cluster_network.cidr }}
hostPrefix: {{ install_config_vars.cluster_network.host_prefix }}
networkType: {{ install_config_vars.cluster_network.type }}
serviceNetwork:
- {{ env.install_config.service_network }}
- {{ install_config_vars.service_network }}
platform:
none: {}
fips: {{ env.install_config.fips }}
fips: {{ install_config_vars.fips }}
pullSecret: '{{ env.redhat.pull_secret if not disconnected.enabled else disconnected.registry.pull_secret }}'
{% if disconnected.enabled %}
{{ 'imageContentSources: ' }}
Expand Down
6 changes: 6 additions & 0 deletions roles/get_ocp/vars/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
install_config_vars: |
{%- if install_config is defined and install_config is iterable -%}
{{ install_config_defaults | combine (install_config, recursive=True) }}
{%- else -%}
{{ install_config_defaults }}
{%- endif -%}
16 changes: 16 additions & 0 deletions roles/prepare_configs/defaults/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Openshift Settings
install_config_defaults:
api_version: v1
compute:
architecture: s390x
hyperthreading: Enabled
control:
architecture: s390x
hyperthreading: Enabled
cluster_network:
cidr: 10.128.0.0/14
host_prefix: 23
type: OVNKubernetes
service_network: 172.30.0.0/16
machine_network: 192.168.122.0/24
fips: 'false'
Loading

0 comments on commit 748bfe3

Please sign in to comment.