Consolidate IRIS and stackhpc branches #160

jovial · 2021-07-14T11:46:55Z

The hope is that we can use a common branch for all deployments. I'm hoping that if we use feature flags for new functionality then we won't need to maintain separate branches.

Depends-on: stackhpc/kolla#80

jovial · 2021-07-14T11:52:27Z

Looking at diff, the prometheus change to v2 looks to be a breaking one. I should hide that behind a feature flag.

cityofships · 2021-07-14T12:39:06Z

Prometheus v2 support already landed in Victoria branch: 9ed79e8

Not controlled via flag, support for v1 is/should be abandoned?

jovial · 2021-07-14T12:45:08Z

Prometheus v2 support already landed in Victoria branch: 9ed79e8

Not controlled via flag, support for v1 is/should be abandoned?

Thanks, I'll drop those patches then :)

jovial · 2021-07-14T12:46:42Z

Can also drop the libvirt exporter stuff once we get:

#156

merged.

jovial · 2021-07-14T12:51:25Z

Prometheus v2 support already landed in Victoria branch: 9ed79e8
Not controlled via flag, support for v1 is/should be abandoned?

Thanks, I'll drop those patches then :)

I've force pushed with the Prometheus patches removed.

cASO is an OpenStack accounting extractor. For more detail see: https://github.com/IFCA/caso By default, cASO is configured to output to Fluentd via TCP. The accounting information can then be shipped off to ElasticSearch. (cherry picked from commit d8ab00f)

Add support for deploying the Kolla Prometheus libvirt exporter image to facilitate gathering metrics from the Nova libvirt service. Partially-Implements: blueprint libvirt-exporter (cherry picked from commit edcd41a) (cherry picked from commit ab84608) Change-Id: Ife0a9ac6e4f03802543d0812d552c30ddb982b9e (cherry picked from commit 1422e0e)

This uses the same approach as the mariadb role (and others). Closes-Bug: : 1928193 Co-Authored-By: John Garbutt <[email protected]> Change-Id: I79a7a8c80327cfd9ef31d17fe71f450a181a638c (cherry picked from commit 502152290ac5fc2a09176c52a459b9741273b752) (cherry picked from commit 9ec49a3)

* Register Swift-compatible endpoints in Keystone * Load balance across RadosGW API servers using HAProxy The support is exercised in the cephadm CI jobs, but since RGW is not currently enabled via cephadm, it is not yet tested. https://docs.ceph.com/en/latest/radosgw/keystone/ Implements: blueprint ceph-rgw Change-Id: I891c3ed4ed93512607afe65a42dd99596fd4dbf9 (cherry picked from commit 5b75fefb7d9f50c197f3cd86e18059cb31ca9984) (cherry picked from commit 31f18de)

This pull request adds support for the OpenID Connect authentication flow in Keystone and enables both ID and access token authentication flows. The ID token configuration is designed to allow users to authenticate via Horizon using an identity federation; whereas the Access token is used to allow users to authenticate in the OpenStack CLI using a federated user. Without this PR, if one wants to configure OpenStack to use identity federation, he/she needs to do a lot of configurations in the keystone, Horizon, and register quite a good number of different parameters using the CLI such as mappings, identity providers, federated protocols, and so on. Therefore, with this PR, we propose a method for operators to introduce/present the IdP's metadata to Kolla-ansible, and based on the presented metadata, Kolla-ansible takes care of all of the configurations to prepare OpenStack to work in a federated environment. Implements: blueprint add-openid-support Co-Authored-By: Jason Anderson <[email protected]> Change-Id: I0203a3470d7f8f2a54d5e126d947f540d93b8210 (cherry picked from commit f3fbe83)

This gets rid of one of the steps in the authentication flow. Closes-Bug: 1930055 Change-Id: I4ed4651b55a912f1d9aec7277bae6bb4776f1e0a (cherry picked from commit 510508e9fa416801af58c1aedcf24e0bf8e88194)

cityofships · 2022-02-02T09:12:23Z

ansible/roles/keystone/templates/wsgi-keystone.conf.j2

@@ -78,6 +78,7 @@ LogLevel info
 {% for idp in keystone_identity_providers %}
 {% if idp.protocol == 'openid' %}
    <LocationMatch /v3/auth/OS-FEDERATION/identity_providers/{{ idp.name }}/protocols/{{ idp.protocol }}/websso>
+      OIDCDiscoverURL {{ keystone_public_url }}/redirect_uri?iss={{ item.identifier | urlencode }}


Should be idp.identifier?

markgoddard · 2022-02-02T17:33:11Z

ansible/roles/caso/meta/main.yml

@@ -0,0 +1,3 @@
+---
+dependencies:
+  - { role: common }


We dropped these role deps in victoria.

markgoddard · 2022-02-04T11:12:24Z

ansible/roles/caso/defaults/main.yml

+####################
+caso_install_type: "{{ kolla_install_type }}"
+caso_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ kolla_base_distro }}-{{ caso_install_type }}-caso"
+caso_tag: "{{ openstack_release }}"


openstack_tag

markgoddard · 2022-02-04T11:16:36Z

ansible/roles/caso/tasks/register.yml

+---
+- name: Configure cASO user
+  kolla_toolbox:
+    module_name: "kolla_keystone_user"


We stopped doing it this way in Train. The module was dropped from the kolla-toolbox image. You need to use service-ks-register role now

markgoddard · 2022-02-04T11:17:46Z

ansible/roles/caso/templates/caso.conf.j2

+
+[keystone_auth]
+auth_type = password
+auth_url = {{ admin_protocol }}://{{ kolla_internal_fqdn }}:{{ keystone_admin_port }}


keystone_admin_url

markgoddard · 2022-02-04T11:21:58Z

ansible/roles/prometheus/tasks/precheck.yml

+
+- name: Checking free ports for Prometheus libvirt-exporter
+  wait_for:
+    host: "{{ hostvars[inventory_hostname]['ansible_' + api_interface]['ipv4']['address'] }}"


'api' | kolla_address

markgoddard · 2022-02-04T11:24:39Z

ansible/roles/prometheus/templates/prometheus.yml.j2

+    static_configs:
+      - targets:
+{% for host in groups["prometheus-libvirt-exporter"] %}
+        - '{{ hostvars[host]['ansible_' + hostvars[host]['api_interface']]['ipv4']['address'] }}:{{ hostvars[host]['prometheus_libvirt_exporter_port'] }}'


- '{{ 'api' | kolla_address(host) | put_address_in_context('url') }}:{{ hostvars[host]['prometheus_libvirt_exporter_port'] }}'

markgoddard · 2022-02-04T11:25:30Z

ansible/roles/prometheus/defaults/main.yml

+####################
+# HAProxy
+####################
+prometheus_haproxy_members: "{% for host in groups['prometheus'] %}server {{ hostvars[host]['ansible_hostname'] }} {{ 'api' | kolla_address(host) }}:{{ prometheus_port }} check inter 2000 rise 2 fall 5{% if not loop.first %} backup{% endif %};{% endfor %}"


ansible_facts.hostname

markgoddard · 2022-02-04T11:28:49Z

ansible/roles/ceph-rgw/defaults/main.yml

+# List of Ceph RadosGW hostname:port to use as HAProxy backends.
+ceph_rgw_hosts: []
+ceph_rgw_haproxy_members: "{{ ceph_rgw_hosts | map('regex_replace', '(.*)', 'server \\1 \\1 ' + ceph_rgw_haproxy_healthcheck) | list }}"
+ceph_rgw_haproxy_healthcheck: "check inter 2000 rise 2 fall 5"


This ended up a bit different upstream:

# List of Ceph hosts to use as HAProxy backends. Each item should contain # 'host' and 'port'` keys. The 'ip' and 'port' keys are optional. If 'ip' is # not specified, the 'host' values should be resolvable from the host running # HAProxy. If the ``port`` is not specified, the default HTTP (80) or HTTPS # (443) port will be used. ceph_rgw_hosts: [] ceph_rgw_haproxy_members: >- {%- set members = [] -%} {%- for host in ceph_rgw_hosts -%} {%- set port = (":" ~ host.port) if host.port is defined else "" -%} {%- set member = "server " ~ host.host ~ " " ~ host.ip | default(host.host) ~ port ~ " " ~ ceph_rgw_haproxy_healthcheck -%} {%- set _ = members.append(member) -%} {%- endfor -%} {{ members }} ceph_rgw_haproxy_healthcheck: "check inter 2000 rise 2 fall 5"

Should we backport again from the merged patch?

markgoddard · 2022-02-04T11:50:52Z

ansible/roles/keystone/templates/wsgi-keystone.conf.j2

@@ -78,6 +78,7 @@ LogLevel info
 {% for idp in keystone_identity_providers %}
 {% if idp.protocol == 'openid' %}
    <LocationMatch /v3/auth/OS-FEDERATION/identity_providers/{{ idp.name }}/protocols/{{ idp.protocol }}/websso>
+      OIDCDiscoverURL {{ keystone_public_url }}/redirect_uri?iss={{ item.identifier | urlencode }}


jovial changed the title ~~Consolidate IRIS and stackhpc branches~~ WIP: Consolidate IRIS and stackhpc branches Jul 14, 2021

jovial force-pushed the cumulus/victoria branch from 267b7fc to aeb41e9 Compare July 14, 2021 12:50

dougszumski and others added 6 commits February 1, 2022 10:11

Add role to deploy cASO

67ee60e

cASO is an OpenStack accounting extractor. For more detail see: https://github.com/IFCA/caso By default, cASO is configured to output to Fluentd via TCP. The accounting information can then be shipped off to ElasticSearch. (cherry picked from commit d8ab00f)

Add OIDCDiscoverURL mod_oidc option

67ccb7e

This gets rid of one of the steps in the authentication flow. Closes-Bug: 1930055 Change-Id: I4ed4651b55a912f1d9aec7277bae6bb4776f1e0a (cherry picked from commit 510508e9fa416801af58c1aedcf24e0bf8e88194)

jovial force-pushed the cumulus/victoria branch from aeb41e9 to 67ccb7e Compare February 1, 2022 10:22

jovial changed the title ~~WIP: Consolidate IRIS and stackhpc branches~~ Consolidate IRIS and stackhpc branches Feb 1, 2022

cityofships reviewed Feb 2, 2022

View reviewed changes

markgoddard suggested changes Feb 4, 2022

View reviewed changes

jovial mentioned this pull request Mar 30, 2022

Add role to deploy cASO #205

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consolidate IRIS and stackhpc branches #160

Consolidate IRIS and stackhpc branches #160

jovial commented Jul 14, 2021 •

edited

Loading

jovial commented Jul 14, 2021

cityofships commented Jul 14, 2021

jovial commented Jul 14, 2021

jovial commented Jul 14, 2021

jovial commented Jul 14, 2021

cityofships Feb 2, 2022

markgoddard Feb 4, 2022

markgoddard Feb 2, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

markgoddard Feb 4, 2022

Consolidate IRIS and stackhpc branches #160

Are you sure you want to change the base?

Consolidate IRIS and stackhpc branches #160

Conversation

jovial commented Jul 14, 2021 • edited Loading

jovial commented Jul 14, 2021

cityofships commented Jul 14, 2021

jovial commented Jul 14, 2021

jovial commented Jul 14, 2021

jovial commented Jul 14, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jovial commented Jul 14, 2021 •

edited

Loading