diff --git a/README.md b/README.md index a5cbee35..6b973398 100644 --- a/README.md +++ b/README.md @@ -99,16 +99,20 @@ We are now ready to install `libvirt` as our hypervisor, provision VMs and prepa Here is an example about [_cluster.yml_](cluster-example.yml) file that contains information about the cluster that is going to be installed. -| variable | describtion | -|---|---| -|cluster_name |Name of the cluster to be installed | -|public_domain |Root domain that will be used for your cluster. | -|ip_families|Decide whether you want IPv4, IPv6 or dual-stack, detault: "['IPv4']"| -|public_ip |Override for public ip entries. defaults to `hostvars['localhost']['ansible_default_ipv4']['address']`. | -|public_ipv6 |Override for public ip entries. defaults to `hostvars['localhost']['ansible_default_ipv6']['address']`. | -|dns_provider |DNS provider, value can be _route53_, _cloudflare_, _gcp_, _azure_,_transip_ or _none_. Check __Setup public DNS records__ for more info. | -|letsencrypt_account_email |Email address that is used to create LetsEncrypt certs. If _cloudflare_account_email_ is not present for CloudFlare DNS recods, _letsencrypt_account_email_ is also used with CloudFlare DNS account email | -|image_pull_secret|Token to be used to authenticate to the Red Hat image registry. You can download your pull secret from https://cloud.redhat.com/openshift/install/metal/user-provisioned | +| variable | description |Default| +|---|---|---| +|`cluster_name` |Name of the cluster to be installed | **Required** | +|`dns_provider` |DNS provider, value can be _route53_, _cloudflare_, _gcp_, _azure_,_transip_ or _none_. Check __Setup public DNS records__ for more info. | **Required** | +|`image_pull_secret` |Token to be used to authenticate to the Red Hat image registry. You can download your pull secret from https://cloud.redhat.com/openshift/install/metal/user-provisioned | **Required** | +|`letsencrypt_account_email` |Email address that is used to create LetsEncrypt certs. If _cloudflare_account_email_ is not present for CloudFlare DNS recods, _letsencrypt_account_email_ is also used with CloudFlare DNS account email | **Required** | +|`public_domain` |Root domain that will be used for your cluster. | **Required** | +|`ip_families` |Decide whether you want IPv4, IPv6 or dual-stack. | `['IPv4']` | +|`listen_address` |Listen address for the load balancer on your host system. |`hostvars['localhost']['ansible_default_ipv4']['address']` | +|`listen_address_ipv6` |Same as listen_address but for IPv6 |`hostvars['localhost']['ansible_default_ipv6']['address']`| +|`public_ip` |Optional to overwrite public ip, if it is different from `listen_address`. Used for dns records at your dns_provider. | `listen_address` | +|`public_ipv6` |Same as `public_ip` but for IPv6 | `listen_address_ipv6` | +|`masters_schedulable` |Optional to overwrite masters schedulable| `false` | +|`sdn_plugin_name` |Optional to change the SDN plugin between `OVNKubernetes` or `OpenShiftSDN` | `OVNKubernetes` | ### Cluster design (single node, compact or normal) @@ -152,8 +156,8 @@ masters_schedulable: false ### Setup public DNS records -Current tools allow use of three DNS providers: _AWS Route53_, _Cloudflare_, _GCP DNS_ or _none_. -If you want to use _Route53_, _Cloudflare_ or _GCP_ as your DNS provider, you have to add a few variables. Check the instructions below. +Current tools allow use of three DNS providers: _AWS Route53_, _Cloudflare_, _DigitalOcean_, _GCP DNS_ or _none_. +If you want to use _Route53_, _Cloudflare_, _DigitalOcean_ or _GCP_ as your DNS provider, you have to add a few variables. Check the instructions below. DNS records are constructed based on _cluster_name_ and _public_domain_ values. With above values DNS records should be - api._cluster_name_._public_domain_ @@ -167,11 +171,12 @@ Please configure in `cluster.yml` all necessary credentials: | DNS provider | Variables | |---|---| +|Azure|`azure_client_id: 'client_id'`
`azure_secret: 'key'`
`azure_subscription_id: 'subscription_id'`
`azure_tenant: 'tenant_id'`
`azure_resource_group: 'dns_zone_resource_group'` | |CloudFlare|`cloudflare_account_email: john@example.com`
Use the global api key here! (API-Token is not supported!) (Details in #86)
`cloudflare_account_api_token: 9348234sdsd894.....`
`cloudflare_zone: domain.tld`| -|Route53 / AWS|`aws_access_key: key`
`aws_secret_key: secret`
`aws_zone: domain.tld`
| +|DigitalOcean|`digitalocean_token: e7a6f82c3245b65cf4.....`
`digitalocean_zone: domain.tld`| |GCP|`gcp_project: project-name `
`gcp_managed_zone_name: 'zone-name'`
`gcp_managed_zone_domain: 'example.com.'`
`gcp_serviceaccount_file: ../gcp_service_account.json` | -|Azure|`azure_client_id: 'client_id'`
`azure_secret: 'key'`
`azure_subscription_id: 'subscription_id'`
`azure_tenant: 'tenant_id'`
`azure_resource_group: 'dns_zone_resource_group'` | |Hetzner|`hetzner_account_api_token: 93543ade82AA$73.....`
`hetzner_zone: domain.tld`| +|Route53 / AWS|`aws_access_key: key`
`aws_secret_key: secret`
`aws_zone: domain.tld`
| |TransIP|`transip_token: eyJ0eXAiOiJKV....`
`transip_zone: domain.tld`| |none|With `dns_provider: none` the playbooks will not create public dns entries. (It will skip letsencrypt too) Please create public dns entries if you want to access your cluster.| @@ -181,6 +186,8 @@ Please configure in `cluster.yml` all necessary credentials: |---|---|---| |`storage_nfs`|false|Setup a local NFS server, create a Storage Class (with [nfs-subdir-external-provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner) ) pointing to it, and use that StorageClass for the internal Registry Storage| |`vm_autostart`|false|Create cluster VMs with `autostart` enabled| +|`vm_storage_backend`|`qcow2`|You can choose between default `qcow2` and `lvm` as storage backend.| +|`vm_storage_backend_location`|empty|Important for vm_storage_backend lvm, please add the volume group for example `vg0`| |`auth_redhatsso`|empty|Install Red Hat SSO, checkout [_cluster-example.yml_](cluster-example.yml) for an example | |`auth_htpasswd`|empty|Install htpasswd, checkout [_cluster-example.yml_](cluster-example.yml) for an example | |`auth_github`|empty|Install GitHub IDP, checkout [_cluster-example.yml_](cluster-example.yml) for an example | @@ -210,6 +217,7 @@ Please configure in `cluster.yml` all necessary credentials: * [Disk management (add disk to vm, wipe node)](docs/disk-management.md) * [How to passthrough nvme or gpu (pci-passthrough](docs/pci-passthrough.md) * [How to install OKD](docs/how-to-install-okd.md) +* [Virsh commands cheatsheet to manage KVM guest virtual machines](https://computingforgeeks.com/virsh-commands-cheatsheet/) # Useful commands @@ -217,3 +225,9 @@ Please configure in `cluster.yml` all necessary credentials: |---|---| |Check haproxy connections| ```podman exec -ti openshift-4-loadbalancer-${cluster_name} ./watch-stats.sh``` |Start cluster after reboot|```./ansible/04-start-cluster.yml``` + + + +# Stargazers over time + +[![Stargazers over time](https://starchart.cc/RedHat-EMEA-SSA-Team/hetzner-ocp4.svg)](https://starchart.cc/RedHat-EMEA-SSA-Team/hetzner-ocp4) diff --git a/ansible/00-provision-hetzner.yml b/ansible/00-provision-hetzner.yml index abe083f7..0a286a87 100755 --- a/ansible/00-provision-hetzner.yml +++ b/ansible/00-provision-hetzner.yml @@ -10,7 +10,7 @@ tasks: - name: Add hetzner server to inventory add_host: - name: "{{ hetzner_hostname }}" + name: "{{ hetzner_ip }}" - name: install hetzner server hosts: all @@ -24,4 +24,3 @@ name: provision-hetzner tags: - provision-hetzner - diff --git a/ansible/roles/letsencrypt/tasks/create-digitalocean.yml b/ansible/roles/letsencrypt/tasks/create-digitalocean.yml new file mode 100644 index 00000000..3a0f50a7 --- /dev/null +++ b/ansible/roles/letsencrypt/tasks/create-digitalocean.yml @@ -0,0 +1,12 @@ +--- + +- name: Create DNS record at DigitalOcean + community.digitalocean.digital_ocean_domain_record: + oauth_token: "{{ digitalocean_token }}" + state: present + domain: "{{ digitalocean_zone }}" + type: TXT + name: "{{ item.0.key | replace(public_domain, '') | regex_replace('\\.$', '') }}" + data: "{{ item.1 }}" + force_update: yes + loop: "{{ challenge_data_dns | default({}) | dict2items | subelements('value') }}" diff --git a/ansible/roles/letsencrypt/tasks/destroy-digitalocean.yml b/ansible/roles/letsencrypt/tasks/destroy-digitalocean.yml new file mode 100644 index 00000000..aa5e01c7 --- /dev/null +++ b/ansible/roles/letsencrypt/tasks/destroy-digitalocean.yml @@ -0,0 +1,12 @@ +--- + +- name: Destroy DNS record at DigitalOcean + community.digitalocean.digital_ocean_domain_record: + oauth_token: "{{ digitalocean_token }}" + state: absent + domain: "{{ digitalocean_zone }}" + type: TXT + name: "{{ item.0.key | replace(public_domain, '') | regex_replace('\\.$', '') }}" + data: "{{ item.1 }}" + force_update: yes + loop: "{{ challenge_data_dns | default({}) | dict2items | subelements('value') }}" diff --git a/ansible/roles/letsencrypt/tasks/main.yml b/ansible/roles/letsencrypt/tasks/main.yml index 2837855e..2b69670c 100644 --- a/ansible/roles/letsencrypt/tasks/main.yml +++ b/ansible/roles/letsencrypt/tasks/main.yml @@ -137,8 +137,6 @@ loop: "{{ challenge_data_dns | default({}) | dict2items | subelements('value') }}" when: le_dns_provider == "transip" and sample_com_challenge is changed - - - name: DNS record info debug: msg: "{{ item.0.key }} TXT {{ item.1 }}" @@ -148,7 +146,7 @@ - name: Include DNS provider include: "create-{{ le_dns_provider }}.yml" when: - - le_dns_provider in ['hetzner'] + - le_dns_provider in ['hetzner', 'digitalocean'] - sample_com_challenge is changed - pause: @@ -232,7 +230,6 @@ loop: "{{ challenge_data_dns | default({}) | dict2items | subelements('value') }}" when: le_dns_provider == "azure" and sample_com_challenge is changed - - name: Delete DNS record at TransIP uri: url: "https://api.transip.nl/v6/domains/{{ transip_zone }}/dns" @@ -251,11 +248,10 @@ loop: "{{ challenge_data_dns | default({}) | dict2items | subelements('value') }}" when: le_dns_provider == "transip" and sample_com_challenge is changed - - name: Include DNS provider include: "destroy-{{ le_dns_provider }}.yml" when: - - le_dns_provider in ['hetzner'] + - le_dns_provider in ['hetzner', 'digitalocean'] - sample_com_challenge is changed - name: concat root ca and intermediate diff --git a/ansible/roles/openshift-4-cluster/defaults/main.yml b/ansible/roles/openshift-4-cluster/defaults/main.yml index 59624f36..f4ad009f 100644 --- a/ansible/roles/openshift-4-cluster/defaults/main.yml +++ b/ansible/roles/openshift-4-cluster/defaults/main.yml @@ -15,6 +15,12 @@ ip_families: listen_address: "{{ hostvars['localhost']['ansible_default_ipv4']['address'] | default('') }}" listen_address_ipv6: "{{ hostvars['localhost']['ansible_default_ipv6']['address'] | default('') }}" +# Default: qcow2 +# For more performance use LVM +vm_storage_backend: qcow2 +# Only necessary for LVM, ignored by qcow but should be '/var/lib/libvirt/images/' +vm_storage_backend_location: + master_count: 3 master_vcpu: 4 master_memory_size: 16384 @@ -36,7 +42,7 @@ vm_autostart: false # Important: OpenShift version must match to RHEL CoreOS version! # reference to OpenShift version -openshift_version: 4.8.2 +openshift_version: 4.9.5 openshift_install_command: "/opt/openshift-install-{{ openshift_version }}/openshift-install" # dev-pre: # {{ openshift_mirror }}/pub/openshift-v4/clients/ocp-dev-preview @@ -53,7 +59,7 @@ opm_download_url: "{{ openshift_location }}/opm-linux-{{ opm_version }}.tar.gz" opm_dest: "/opt/openshift-client-{{ openshift_client_version }}/" # reference to coreos qcow file -coreos_version: 4.8.2 +coreos_version: 4.9.0 coreos_download_url: "{{ openshift_mirror }}/pub/openshift-v4/dependencies/rhcos/{{ coreos_version.split('.')[:2]|join('.') }}/{{ coreos_version }}/rhcos-{{coreos_version}}-x86_64-qemu.x86_64.qcow2.gz" coreos_csum_url: "{{ openshift_mirror }}/pub/openshift-v4/dependencies/rhcos/{{ coreos_version.split('.')[:2]|join('.') }}/{{ coreos_version }}/sha256sum.txt" diff --git a/ansible/roles/openshift-4-cluster/tasks/create-network.yml b/ansible/roles/openshift-4-cluster/tasks/create-network.yml index 4249e05a..da37ec54 100644 --- a/ansible/roles/openshift-4-cluster/tasks/create-network.yml +++ b/ansible/roles/openshift-4-cluster/tasks/create-network.yml @@ -15,10 +15,11 @@ set_fact: vn_subnet_ipv6: "{{ hostvars['localhost']['ansible_default_ipv6']['address'].split(':')[:4] | join(':') | string}}:{{ '%x' % vn_subnet.split('.')[2] | int }}" ipv6_listen_public: - - "{{ public_ipv6 | default(listen_address_ipv6) }}" + - "{{ listen_address_ipv6 }}" ipv6_listen_private: - "{{ hostvars['localhost']['ansible_default_ipv6']['address'].split(':')[:4] | join(':') | string}}:{{ '%x' % vn_subnet.split('.')[2] | int }}::1" when: "'IPv6' in ip_families" + tags: always - name: Build IPv4 subnet set_fact: @@ -26,8 +27,9 @@ ipv4_listen_private: - "{{ vn_subnet.split('.')[:3] | join('.')}}.1" ipv4_listen_public: - - "{{ public_ip | default(listen_address) }}" + - "{{ listen_address }}" when: "'IPv4' in ip_families" + tags: always - name: Build list of nodes set_fact: @@ -157,6 +159,5 @@ path: /etc/hosts marker: "# {mark} ANSIBLE MANAGED BLOCK {{ cluster_name }}.{{ public_domain }}" block: | - {{ public_ip | default(listen_address) }} api.{{ cluster_name }}.{{ public_domain }} + {{ listen_address }} api.{{ cluster_name }}.{{ public_domain }} tags: public_dns - when: dns_provider == 'none' \ No newline at end of file diff --git a/ansible/roles/openshift-4-cluster/tasks/create-vm.yml b/ansible/roles/openshift-4-cluster/tasks/create-vm.yml index d2992bd7..25cdfec1 100644 --- a/ansible/roles/openshift-4-cluster/tasks/create-vm.yml +++ b/ansible/roles/openshift-4-cluster/tasks/create-vm.yml @@ -6,17 +6,36 @@ command: "qemu-img create -f qcow2 -b {{ coreos_image_location }} /var/lib/libvirt/images/{{ vm_instance_name }}.qcow2 {{ vm_root_disk_size }}" args: creates: "/var/lib/libvirt/images/{{ vm_instance_name }}.qcow2" + when: vm_storage_backend == "qcow2" + +- name: Convert coreos qcow into raw image + command: "qemu-img convert {{ coreos_image_location }} -O raw {{ coreos_image_location }}.raw " + args: + creates: "{{ coreos_image_location }}.raw" + when: vm_storage_backend == "lvm" + +- name: Create logical volume + lvol: + vg: "{{ vm_storage_backend_location }}" + lv: "{{ vm_instance_name }}" + state: present + size: "{{ vm_root_disk_size }}" + when: vm_storage_backend == "lvm" + +- name: Copy/dd coreos into device + command: "dd if={{ coreos_image_location }}.raw of=/dev/{{ vm_storage_backend_location }}/{{ vm_instance_name }} bs=4M" + when: vm_storage_backend == "lvm" - name: Copy ignition {{ vm_instance_name }} copy: src: "{{ vm_ignition_file }}" dest: "/var/lib/libvirt/images/{{ vm_instance_name }}.ign" mode: '0644' - -# - name: Debug - create /tmp/{{ vm_instance_name }}.virt.xml -# template: -# src: "vm.xml.j2" -# dest: "/tmp/{{ vm_instance_name }}.virt.xml" + +- name: Debug - create /tmp/{{ vm_instance_name }}.virt.xml + template: + src: "vm.xml.j2" + dest: "/tmp/{{ vm_instance_name }}.virt.xml" - name: Define VirtualMachine {{ vm_instance_name }} virt: @@ -25,7 +44,7 @@ xml: "{{ lookup('template', 'templates/vm.xml.j2') }}" - name: Start VirtualMachine {{ vm_instance_name }} - virt: + virt: name: "{{ vm_instance_name }}" state: running autostart: "{{ vm_autostart }}" diff --git a/ansible/roles/openshift-4-cluster/tasks/create.yml b/ansible/roles/openshift-4-cluster/tasks/create.yml index 46daa444..9a08378d 100644 --- a/ansible/roles/openshift-4-cluster/tasks/create.yml +++ b/ansible/roles/openshift-4-cluster/tasks/create.yml @@ -65,6 +65,9 @@ le_hetzner_account_api_token: "{{ hetzner_account_api_token }}" le_hetzner_zone: "{{ hetzner_zone }}" + le_digitalocean_token: "{{ digitalocean_token }}" + le_digitalocean_zone: "{{ digitalocean_zone }}" + tags: letsencrypt when: letsencrypt_disabled == false diff --git a/ansible/roles/openshift-4-cluster/tasks/destroy-network.yml b/ansible/roles/openshift-4-cluster/tasks/destroy-network.yml index 267badd9..5a3c6df4 100644 --- a/ansible/roles/openshift-4-cluster/tasks/destroy-network.yml +++ b/ansible/roles/openshift-4-cluster/tasks/destroy-network.yml @@ -5,7 +5,8 @@ tasks_from: destroy.yml vars: pd_provider: "{{ dns_provider }}" - pd_public_ip: "{{ listen_address }}" + pd_public_ip: "{% if 'IPv4' in ip_families %}{{ public_ip | default(listen_address) }}{% endif %}" + pd_public_ipv6: "{% if 'IPv6' in ip_families %}{{ public_ipv6 | default(listen_address_ipv6) }}{% endif %}" pd_cloudflare_account_api_token: "{{ cloudflare_account_api_token }}" pd_cloudflare_zone: "{{ cloudflare_zone }}" pd_aws_access_key: "{{ aws_access_key }}" @@ -23,7 +24,6 @@ marker: "# {mark} ANSIBLE MANAGED BLOCK {{ cluster_name }}.{{ public_domain }}" block: "" tags: public_dns - when: dns_provider == 'none' - name: Destroy OpenShift 4 load balancer import_role: diff --git a/ansible/roles/openshift-4-cluster/tasks/destroy-vm.yml b/ansible/roles/openshift-4-cluster/tasks/destroy-vm.yml index a50e9c79..967ef5ee 100644 --- a/ansible/roles/openshift-4-cluster/tasks/destroy-vm.yml +++ b/ansible/roles/openshift-4-cluster/tasks/destroy-vm.yml @@ -11,10 +11,19 @@ name: "{{ vm_instance_name }}" ignore_errors: yes +- name: Delete logical volume + lvol: + vg: "{{ vm_storage_backend_location }}" + lv: "{{ vm_instance_name }}" + state: absent + force: yes + when: vm_storage_backend == "lvm" + - name: Delete disk {{ vm_instance_name }} file: path: "/var/lib/libvirt/images/{{ vm_instance_name }}.qcow2" state: absent + when: vm_storage_backend == "qcow2" - name: Delete ignition {{ vm_instance_name }} file: diff --git a/ansible/roles/openshift-4-cluster/tasks/post-install-storage-nfs.yml b/ansible/roles/openshift-4-cluster/tasks/post-install-storage-nfs.yml index dee35dae..3a060376 100644 --- a/ansible/roles/openshift-4-cluster/tasks/post-install-storage-nfs.yml +++ b/ansible/roles/openshift-4-cluster/tasks/post-install-storage-nfs.yml @@ -36,8 +36,15 @@ group: root mode: 0644 block: | + {% if 'IPv4' in ip_families -%} {{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-infra-registry {{ vn_subnet }}/24(rw,sync,no_root_squash) {{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-user-pvs {{ vn_subnet }}/24(rw,sync,no_root_squash) + {% endif -%} + {% if 'IPv6' in ip_families -%} + {{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-infra-registry {{ vn_subnet_ipv6 }}::/80(rw,sync,no_root_squash) + {{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-user-pvs {{ vn_subnet_ipv6 }}::/80(rw,sync,no_root_squash) + {% endif -%} + tags: exports - name: Adjust directory permissions @@ -74,7 +81,7 @@ storage: 100Gi nfs: path: "{{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-infra-registry" - server: "{{ vn_subnet.split('.')[:3] | join('.')}}.1" + server: "host.compute.local" persistentVolumeReclaimPolicy: Recycle - name: Create registry-storage pvc @@ -277,13 +284,13 @@ - name: PROVISIONER_NAME value: redhat-emea-ssa-team/hetzner-ocp4 - name: NFS_SERVER - value: "{{ vn_subnet.split('.')[:3] | join('.')}}.1" + value: "host.compute.local" - name: NFS_PATH value: "{{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-user-pvs" volumes: - name: nfs-client-root nfs: - server: "{{ vn_subnet.split('.')[:3] | join('.')}}.1" + server: "host.compute.local" path: "{{ storage_nfs_path_prefix }}/{{ cluster_name }}-pv-user-pvs" - name: Storage Class diff --git a/ansible/roles/openshift-4-cluster/tasks/prepare-host.yml b/ansible/roles/openshift-4-cluster/tasks/prepare-host.yml index 783e4fe7..61c50229 100644 --- a/ansible/roles/openshift-4-cluster/tasks/prepare-host.yml +++ b/ansible/roles/openshift-4-cluster/tasks/prepare-host.yml @@ -38,3 +38,7 @@ insertafter: '^\[Network\]' regexp: '^IPForward=' when: stat_result.stat.exists == True + +# Install ansible collection for digitalocean dns provider +- name: Ansible collection community.digitalocean + command: "ansible-galaxy collection install community.digitalocean" diff --git a/ansible/roles/openshift-4-cluster/templates/network.xml.j2 b/ansible/roles/openshift-4-cluster/templates/network.xml.j2 index 28a6cc35..f115d1b4 100644 --- a/ansible/roles/openshift-4-cluster/templates/network.xml.j2 +++ b/ansible/roles/openshift-4-cluster/templates/network.xml.j2 @@ -8,9 +8,8 @@ host.{{ vn_internal_domain }} api-int.{{ vn_public_domain }} api.{{ vn_public_domain }} - {% if dns_provider == 'none' %} oauth-openshift.apps.{{ vn_public_domain }} - {% endif %} + console-openshift-console.apps.{{ vn_public_domain }} {% endif %} {% if 'IPv6' in ip_families %} @@ -18,9 +17,8 @@ host.{{ vn_internal_domain }} api-int.{{ vn_public_domain }} api.{{ vn_public_domain }} - {% if dns_provider == 'none' %} oauth-openshift.apps.{{ vn_public_domain }} - {% endif %} + console-openshift-console.apps.{{ vn_public_domain }} {% endif %} diff --git a/ansible/roles/openshift-4-cluster/templates/vm.xml.j2 b/ansible/roles/openshift-4-cluster/templates/vm.xml.j2 index 8b088072..5242dd6e 100644 --- a/ansible/roles/openshift-4-cluster/templates/vm.xml.j2 +++ b/ansible/roles/openshift-4-cluster/templates/vm.xml.j2 @@ -33,11 +33,21 @@ /usr/libexec/qemu-kvm +{% if vm_storage_backend is defined and vm_storage_backend == "lvm" %} + + + + + +{% else %} +{% endif %} + + diff --git a/ansible/roles/provision-hetzner/defaults/main.yml b/ansible/roles/provision-hetzner/defaults/main.yml index 11f2d4d7..5f9298c6 100644 --- a/ansible/roles/provision-hetzner/defaults/main.yml +++ b/ansible/roles/provision-hetzner/defaults/main.yml @@ -1,10 +1,11 @@ --- -hetzner_autosetup_file: autosetup -hetzner_disk1: nvme0n1 -hetzner_disk2: nvme1n1 -hetzner_image: "/root/.oldroot/nfs/install/../images/CentOS-82-64-minimal.tar.gz" +hetzner_webservice_username: # see Hetzner Robot -> Settings -> Webservice +hetzner_webservice_password: +hetzner_hostname: # needed for Hetzner server computer name +hetzner_ip: # needed for Hetzner Web UI and Ansible +hetzner_autosetup_file: templates/autosetup +hetzner_disk1: sda +hetzner_disk2: sdb +hetzner_image: "/root/.oldroot/nfs/install/../images/CentOS-84-64-minimal.tar.gz" hetzner_image_ignore_errors: false -# hetzner_webservice_username: -# hetzner_webservice_password: -# hetzner_hostname: "hostname.example.com" - +ansible_python_interpreter: /usr/bin/python3 # target host python interpreter diff --git a/ansible/roles/public_dns/tasks/create-digitalocean.yml b/ansible/roles/public_dns/tasks/create-digitalocean.yml new file mode 100644 index 00000000..4a4b9d75 --- /dev/null +++ b/ansible/roles/public_dns/tasks/create-digitalocean.yml @@ -0,0 +1,32 @@ +--- + +- name: Create DNS record at DigitalOcean + community.digitalocean.digital_ocean_domain_record: + oauth_token: "{{ digitalocean_token }}" + state: present + domain: "{{ digitalocean_zone }}" + type: A + name: "{{ item }}.{{ cluster_name }}" + data: "{{ pd_public_ip }}" + force_update: yes + with_items: + - api + - '*.apps' + tags: + - public_dns + +- name: Create DNS record at DigitalOcean IPv6 + community.digitalocean.digital_ocean_domain_record: + oauth_token: "{{ digitalocean_token }}" + state: present + domain: "{{ digitalocean_zone }}" + type: AAAA + name: "{{ item }}.{{ cluster_name }}" + data: "{{ pd_public_ipv6 }}" + force_update: yes + with_items: + - api + - '*.apps' + tags: + - public_dns + diff --git a/ansible/roles/public_dns/tasks/create-transip.yml b/ansible/roles/public_dns/tasks/create-transip.yml index e806acf0..c43415da 100644 --- a/ansible/roles/public_dns/tasks/create-transip.yml +++ b/ansible/roles/public_dns/tasks/create-transip.yml @@ -17,4 +17,27 @@ - '*.apps' tags: - public_dns + when: (pd_public_ip is defined) and (pd_public_ip|length > 0) + + +- name: Create IPv6 DNS record at TransIP + uri: + url: "https://api.transip.nl/v6/domains/{{ transip_zone }}/dns" + method: POST + headers: + Authorization: "Bearer {{ transip_token }}" + body_format: json + body: + dnsEntry: + name: "{{ item }}.{{ cluster_name }}" + expire: 60 + type: AAAA + content: "{{ pd_public_ipv6 }}" + status_code: 201 + with_items: + - api + - '*.apps' + tags: + - public_dns + when: (pd_public_ipv6 is defined) and (pd_public_ipv6|length > 0) diff --git a/ansible/roles/public_dns/tasks/destroy-digitalocean.yml b/ansible/roles/public_dns/tasks/destroy-digitalocean.yml new file mode 100644 index 00000000..9b30c028 --- /dev/null +++ b/ansible/roles/public_dns/tasks/destroy-digitalocean.yml @@ -0,0 +1,32 @@ +--- + +- name: Destroy DNS record at DigitalOcean + community.digitalocean.digital_ocean_domain_record: + oauth_token: "{{ digitalocean_token }}" + state: absent + domain: "{{ digitalocean_zone }}" + type: A + name: "{{ item }}.{{ cluster_name }}" + data: "{{ pd_public_ip }}" + force_update: yes + with_items: + - api + - '*.apps' + tags: + - public_dns + +- name: Destroy DNS record at DigitalOcean IPv6 + community.digitalocean.digital_ocean_domain_record: + oauth_token: "{{ digitalocean_token }}" + state: absent + domain: "{{ digitalocean_zone }}" + type: AAAA + name: "{{ item }}.{{ cluster_name }}" + data: "{{ pd_public_ipv6 }}" + force_update: yes + with_items: + - api + - '*.apps' + tags: + - public_dns + diff --git a/ansible/roles/public_dns/tasks/destroy-transip.yml b/ansible/roles/public_dns/tasks/destroy-transip.yml index 7c92d006..5d06ed8a 100644 --- a/ansible/roles/public_dns/tasks/destroy-transip.yml +++ b/ansible/roles/public_dns/tasks/destroy-transip.yml @@ -1,4 +1,4 @@ -- name: Create DNS record at TransIP +- name: Delete DNS record at TransIP uri: url: "https://api.transip.nl/v6/domains/{{ transip_zone }}/dns" method: DELETE @@ -17,4 +17,27 @@ - '*.apps' tags: - public_dns + when: (pd_public_ip is defined) and (pd_public_ip | length > 0) + +- name: Delete IPv6 DNS record at TransIP + uri: + url: "https://api.transip.nl/v6/domains/{{ transip_zone }}/dns" + method: DELETE + headers: + Authorization: "Bearer {{ transip_token }}" + body_format: json + body: + dnsEntry: + name: "{{ item }}.{{ cluster_name }}" + expire: 60 + type: AAAA + content: "{{ pd_public_ipv6 }}" + status_code: 204 + with_items: + - api + - '*.apps' + tags: + - public_dns + when: (pd_public_ipv6 is defined) and (pd_public_ipv6|length > 0) + diff --git a/docs/dns-forwarder/dns-create.yml b/docs/dns-forwarder/dns-create.yml index a69b201e..ca593483 100755 --- a/docs/dns-forwarder/dns-create.yml +++ b/docs/dns-forwarder/dns-create.yml @@ -1,4 +1,4 @@ -#!/usr/bin/env ansible-playbook +#!/usr/bin/env ansible-playbook --- - hosts: localhost connection: local @@ -10,7 +10,7 @@ - ../../cluster.yml vars: centos_download_url: https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.1.1911-20200113.3.x86_64.qcow2 - centos_image_location: /var/lib/libvirt/images/CentOS-8-GenericCloud-8.1.1911-20200113.3.x86_64.qcow2 + centos_image_location: /var/lib/libvirt/images/CentOS-8-GenericCloud-8.1.1911-20200113.3.x86_64.qcow2 # centos_download_url: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 # centos_image_location: /var/lib/libvirt/images/CentOS-7-x86_64-GenericCloud.qcow2 @@ -23,11 +23,6 @@ creates: "{{ centos_image_location }}" warn: false - - name: Install cloud-utils - package: - name: cloud-utils - - - name: Build inventory add_host: hostname: "{{ item.name }}" @@ -45,6 +40,7 @@ vm_memory_unit: 'MiB' groups: - dns + loop: - { name: "{{ cluster_name }}-dns-1", vm_ip_address: "{{ vn_subnet.split('.')[:3] | join('.')}}.2" } - { name: "{{ cluster_name }}-dns-2", vm_ip_address: "{{ vn_subnet.split('.')[:3] | join('.')}}.3" } @@ -52,21 +48,26 @@ - hosts: dns gather_facts: false become: true - tasks: + tasks: ######################################################################################################### # Spinup instances ######################################################################################################### - name: Spinn up instances connection: local - block: + block: - name: Create disk for {{ vm_instance_name }} command: "qemu-img create -f qcow2 -b {{ centos_image_location }} /var/lib/libvirt/images/{{ vm_instance_name }}.qcow2 {{ vm_root_disk_size }}" args: creates: "/var/lib/libvirt/images/{{ vm_instance_name }}.qcow2" + - name: Creates directory + file: + path: "/var/lib/libvirt/images/{{ vm_instance_name }}" + state: directory + - name: Create cloud-init meta-data copy: - dest: "/var/lib/libvirt/images/{{ vm_instance_name }}.meta-data" + dest: "/var/lib/libvirt/images/{{ vm_instance_name }}/meta-data" content: | instance-id: {{ lookup('lines','uuidgen') }} local-hostname: {{ vm_instance_name }} @@ -84,18 +85,19 @@ # Root pw r3dh4t1! - name: Create cloud-init user-data copy: - dest: "/var/lib/libvirt/images/{{ vm_instance_name }}.user-data" + dest: "/var/lib/libvirt/images/{{ vm_instance_name }}/user-data" content: | #cloud-config ssh_authorized_keys: - {{ ssh_public_key }} + hostname: {{ vm_instance_name }} fqdn: {{ vm_instance_name }} # set the locale locale: en_US.UTF-8 - + # timezone: set the timezone for this instance timezone: UTC @@ -107,14 +109,16 @@ sudo: ALL=(ALL) NOPASSWD:ALL ssh_authorized_keys: - {{ ssh_public_key }} + lock_passwd: false passwd: $6$BH7mErbrdppnmXf1$kHwl49XSEvLPSKgeFaJAArjwjxfnMdDbT1DwHykkJr/W9WYJCoHu7nOg1mAWFtY29X.ZgLrzthXA.j71mLugB0 - name: Create CD-ROM ISO with cloud-init config shell: | - cloud-localds /var/lib/libvirt/images/{{ vm_instance_name }}-cloud-init.iso \ - /var/lib/libvirt/images/{{ vm_instance_name }}.user-data \ - /var/lib/libvirt/images/{{ vm_instance_name }}.meta-data && + genisoimage -output /var/lib/libvirt/images/{{ vm_instance_name }}-cloud-init.iso \ + -volid cidata -joliet -rock -graft-points \ + /var/lib/libvirt/images/{{ vm_instance_name }}/user-data \ + /var/lib/libvirt/images/{{ vm_instance_name }}/meta-data && chown qemu:wheel /var/lib/libvirt/images/{{ vm_instance_name }}-cloud-init.iso args: creates: "/var/lib/libvirt/images/{{ vm_instance_name }}-cloud-init.iso" @@ -196,7 +200,7 @@ - name: Start VirtualMachine {{ vm_instance_name }} - virt: + virt: name: "{{ vm_instance_name }}" state: running - name: Wait 300 seconds for port 22 to become open @@ -215,7 +219,7 @@ become: true tags: - vm - tasks: + tasks: - name: Setup nameserver copy: @@ -231,7 +235,7 @@ - name: Install dnsmasq yum: - name: + name: - dnsmasq - bind-utils diff --git a/docs/dns-forwarder/dns-destroy.yml b/docs/dns-forwarder/dns-destroy.yml index 5e88968f..aaad3873 100755 --- a/docs/dns-forwarder/dns-destroy.yml +++ b/docs/dns-forwarder/dns-destroy.yml @@ -1,4 +1,4 @@ -#!/usr/bin/env ansible-playbook +#!/usr/bin/env ansible-playbook --- - hosts: localhost connection: local @@ -25,13 +25,16 @@ name: "{{ vm_instance_name }}" ignore_errors: yes + - name: Delete directory + file: + state: absent + path: "/var/lib/libvirt/images/{{ vm_instance_name }}" + - name: Delete files of {{ vm_instance_name }} file: path: "{{ item }}" state: absent with_items: - "/var/lib/libvirt/images/{{ vm_instance_name }}.qcow2" - - "/var/lib/libvirt/images/{{ vm_instance_name }}.user-data" - - "/var/lib/libvirt/images/{{ vm_instance_name }}.meta-data" - "/var/lib/libvirt/images/{{ vm_instance_name }}-cloud-init.iso" diff --git a/docs/hetzner.md b/docs/hetzner.md index 22806a37..6c1eede2 100644 --- a/docs/hetzner.md +++ b/docs/hetzner.md @@ -123,19 +123,22 @@ You are now ready to reboot your system into the newly installed OS. If you do not want to do the above steps by hand: use Ansible! :-) -1) Create `cluster.yml` with some hetzner specific options: +1) Create a `cluster.yml` in the repo's root folder and add your Hetzner server specifc information, as in the following example: ``` - # Hetzner informations (for role provision-hetzner) - hetzner_hostname: "hostname.domain.tld" - hetzner_webservice_username: "xxxx" - hetzner_webservice_password: "xxxx" - hetzner_ip: "xxx.xxx.xxx.xxx" - hetzner_disk1: nvme0n1 - hetzner_disk1: nvme1n1 - - # Optional: - # hetzner_image: "/root/.oldroot/nfs/install/../images/CentOS-75-64-minimal.tar.gz" - # hetzner_autosetup_file: "{{ playbook_dir }}/my-autosetup-for-openshift" + hetzner_webservice_username: "USERNAME" + hetzner_webservice_password: "PASSWORD" + hetzner_hostname: "HOSTNAME" + hetzner_ip: "IP_ADDRESS" ``` -2) Run playbook: `./ansible/00-provision-hetzner.yml` + More detailed information about the configurable parameters can be found in the [defaults variable file](../ansible/roles/provision-hetzner/defaults/main.yml) + +2) Run playbook: `ansible/00-provision-hetzner.yml` + + Assuming you protected the SSH private key for logging into your Hetzner server with a password, you should use the ssh-agent for calling the Ansible playbook, so that the password for the SSH key is requested once and then stored for the entire session (so that you don't need to enter your password multiple times). The full playbook call from the repo's root folder (using the standard private SSH key name `id_rsa`) would then look like this: + + ``` + ssh-agent bash + ssh-add ~/.ssh/id_rsa + ansible-playbook ansible/00-provision-hetzner.yml + ``` diff --git a/docs/how-to-install-okd.md b/docs/how-to-install-okd.md index 5aa08fbf..f173fa9d 100644 --- a/docs/how-to-install-okd.md +++ b/docs/how-to-install-okd.md @@ -7,14 +7,16 @@ image_pull_secret: '{"auths":{"fake":{"auth": "bar"}}}' cluster_name: okd -openshift_version: 4.7.0-0.okd-2021-07-03-190901 -openshift_location: "https://github.com/openshift/okd/releases/download/{{ openshift_version }}" -coreos_version: 34.20210611.3.0 -coreos_download_url: "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/{{ coreos_version }}/x86_64/fedora-coreos-{{ coreos_version }}-qemu.x86_64.qcow2.xz" -coreos_csum_str: 9a362a7b13e213d8fb01d4c371c3f99893e6a418a60eff650b734c9683f1b06f +openshift_version: 4.8.0-0.okd-2021-10-01-221835 +openshift_location: https://github.com/openshift/okd/releases/download/{{ openshift_version }} +coreos_version: 34.20210904.3.0 +coreos_download_url: https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/{{ coreos_version }}/x86_64/fedora-coreos-{{ coreos_version }}-qemu.x86_64.qcow2.xz +coreos_csum_str: 58e9e841a42944c616925deaa11849881c9cfc47e7d2116bc3c2c9985fea632a - -opm_version: "1.17.4" +opm_version: "1.19.0" opm_download_url: "https://github.com/operator-framework/operator-registry/releases/download/v{{ opm_version }}/linux-amd64-opm" opm_dest: "/opt/operator-registry-{{ opm_version }}/" -``` \ No newline at end of file +``` + +Instead of the above image_pull_secret, it can also be included the same one that would be required for the Openshift Installation. This would allow for later enablement of CatalogSources redhat-marketplace, redhat-operators and certified-operators. +However, after cluster installation, they must be disabled and enabled again. \ No newline at end of file diff --git a/docs/release-notes.md b/docs/release-notes.md index 44776633..6ec41061 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -1,5 +1,32 @@ # RELEASE NOTES +## 2021-12-17 + + * **Bump OpenShift Version to 4.9.5** + * Refactor DigitalOcean DNS provider + * Added IPv6 support + * Switch to ansible galaxy module + Please run `./ansible/setup.yml` or `./ansible/01-prepare-host.yml` to install ansible galaxy module. Or via: `ansible-galaxy collection install community.digitalocean` + * Fixed Issue #185 : IPv6 Single Stack - NFS exports only for IPv4 -> installation fail / not completed + * Fixed Issue #197 : public_ip & listen_address did not work as expected + * Add support for different vm storage backend (lvm & qcow2) + * Added api. to /etc/hosts to be more independent from public DNS + * Added openshift console and oauth url to internal dns entries + * Update documentation + * Redesign the variables table + * Added link to virt cheatsheet + * Bump OKD version to 4.8 & OPM to 1.19 (docs only) + + +## 2021-07-27 + + * **Bump OpenShift version to 4.8.2** + * Added TransIP dns provider with #177 + * Added Rocky basis installation with #181 + * Added IPv6 support #182 + * Tested and documented single node installation #176 + * Fixed NFS provisioning #175 + ## 2021-04-09 * Bump OpenShift Version to 4.7.0