Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OVN LB support #687

Merged
merged 3 commits into from
Feb 26, 2024
Merged

OVN LB support #687

merged 3 commits into from
Feb 26, 2024

Conversation

chess-knight
Copy link
Member

@chess-knight chess-knight commented Feb 5, 2024

All the upstream issues (kubernetes/cloud-provider-openstack#2225, kubernetes/cloud-provider-openstack#2489, osism/issues#499) were resolved, we do not need --allow-preview-features for the OVN load balancer in front of ingress nginx.
Related to #407
Fixes #672
Fixes SovereignCloudStack/issues#298

close kubernetes/cloud-provider-openstack#2234 also after this is merged

use_ovn_lb_provider = "true"

@scs-zuul
Copy link

scs-zuul bot commented Feb 5, 2024

Build canceled (e2e-quick-test pipeline).

@scs-zuul
Copy link

scs-zuul bot commented Feb 5, 2024

Build succeeded (e2e-quick-test pipeline).
https://zuul.scs.community/t/SCS/buildset/2a908b55a8444818a642f6d916c12b42

✔️ k8s-cluster-api-provider-e2e-quick SUCCESS in 25m 59s

Warning:

Sonobouy results === Collecting results === time="2024-02-05T15:15:23Z" level=info msg="delete request issued" dry-run=false kind=namespace namespace=sonobuoy time="2024-02-05T15:15:23Z" level=info msg="delete request issued" dry-run=false kind=clusterrolebindings names="[sonobuoy-serviceaccount-sonobuoy]" time="2024-02-05T15:15:23Z" level=info msg="delete request issued" dry-run=false kind=clusterroles names="[sonobuoy-serviceaccount-sonobuoy]" Plugin: e2e Status: passed Total: 7213 Passed: 5 Failed: 0 Skipped: 7208

Plugin: systemd-logs
Status: passed
Total: 4
Passed: 4
Failed: 0
Skipped: 0

Run Details:
API Server version: v1.27.8
Node health: 4/4 (100%)
Pods health: 33/33 (100%)
Errors detected in files:
Errors:
79 podlogs/kube-system/kube-scheduler-pr687-8d2c78-59l92-cmkvl/logs/kube-scheduler.txt
71 podlogs/kube-system/kube-controller-manager-pr687-8d2c78-59l92-cmkvl/logs/kube-controller-manager.txt
19 podlogs/kube-system/snapshot-controller-55687d7977-lpw7s/logs/snapshot-controller.txt
13 podlogs/kube-system/kube-apiserver-pr687-8d2c78-59l92-cmkvl/logs/kube-apiserver.txt
12 podlogs/kube-system/openstack-cloud-controller-manager-rn5r9/logs/openstack-cloud-controller-manager.txt
11 podlogs/kube-system/cilium-k8sxr/logs/cilium-agent.txt
8 podlogs/kube-system/cilium-l59c7/logs/cilium-agent.txt
8 podlogs/kube-system/cilium-wln8j/logs/cilium-agent.txt
7 podlogs/kube-system/cilium-mvxwr/logs/cilium-agent.txt
2 podlogs/kube-system/cilium-operator-788c7d7585-kgh8w/logs/cilium-operator.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-zj2vq/logs/node-driver-registrar.txt
1 podlogs/sonobuoy/sonobuoy-e2e-job-4e36aabb5f044f54/logs/e2e.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-2kv8x/logs/node-driver-registrar.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-995dr/logs/node-driver-registrar.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-hs9sx/logs/node-driver-registrar.txt
1 podlogs/kube-system/etcd-pr687-8d2c78-59l92-cmkvl/logs/etcd.txt
Warnings:
374 podlogs/kube-system/etcd-pr687-8d2c78-59l92-cmkvl/logs/etcd.txt
36 podlogs/kube-system/kube-apiserver-pr687-8d2c78-59l92-cmkvl/logs/kube-apiserver.txt
34 podlogs/kube-system/kube-scheduler-pr687-8d2c78-59l92-cmkvl/logs/kube-scheduler.txt
16 podlogs/kube-system/openstack-cloud-controller-manager-rn5r9/logs/openstack-cloud-controller-manager.txt
11 podlogs/kube-system/kube-controller-manager-pr687-8d2c78-59l92-cmkvl/logs/kube-controller-manager.txt
6 podlogs/kube-system/csi-cinder-nodeplugin-2kv8x/logs/node-driver-registrar.txt
6 podlogs/kube-system/csi-cinder-nodeplugin-hs9sx/logs/node-driver-registrar.txt
6 podlogs/kube-system/cilium-k8sxr/logs/cilium-agent.txt
5 podlogs/kube-system/csi-cinder-controllerplugin-7b6f6db96b-845ql/logs/csi-attacher.txt
4 podlogs/kube-system/csi-cinder-controllerplugin-7b6f6db96b-845ql/logs/csi-provisioner.txt
4 podlogs/kube-system/cilium-l59c7/logs/cilium-agent.txt
4 podlogs/kube-system/cilium-wln8j/logs/cilium-agent.txt
3 podlogs/kube-system/cilium-mvxwr/logs/cilium-agent.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-zj2vq/logs/node-driver-registrar.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-995dr/logs/liveness-probe.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-zj2vq/logs/liveness-probe.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-2kv8x/logs/liveness-probe.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-995dr/logs/node-driver-registrar.txt
1 podlogs/sonobuoy/sonobuoy-e2e-job-4e36aabb5f044f54/logs/e2e.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-hs9sx/logs/liveness-probe.txt
1 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt
1 podlogs/kube-system/metrics-server-5fb5598cf8-4wvbq/logs/metrics-server.txt
time="2024-02-05T15:15:24Z" level=info msg="delete request issued" dry-run=false kind=namespace namespace=sonobuoy
time="2024-02-05T15:15:24Z" level=info msg="delete request issued" dry-run=false kind=clusterrolebindings names="[]"
time="2024-02-05T15:15:24Z" level=info msg="delete request issued" dry-run=false kind=clusterroles names="[]"

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[]}

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[{Type:NamespaceDeletionDiscoveryFailure Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ResourcesDiscovered Message:All resources successfully discovered} {Type:NamespaceDeletionGroupVersionParsingFailure Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ParsedGroupVersions Message:All legacy kube types successfully parsed} {Type:NamespaceDeletionContentFailure Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ContentDeleted Message:All content successfully deleted, may be waiting on finalization} {Type:NamespaceContentRemaining Status:True LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:SomeResourcesRemain Message:Some resources are remaining: pods. has 5 resource instances} {Type:NamespaceFinalizersRemaining Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ContentHasNoFinalizers Message:All content-preserving finalizers finished}]}

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[{Type:NamespaceDeletionDiscoveryFailure Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ResourcesDiscovered Message:All resources successfully discovered} {Type:NamespaceDeletionGroupVersionParsingFailure Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ParsedGroupVersions Message:All legacy kube types successfully parsed} {Type:NamespaceDeletionContentFailure Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ContentDeleted Message:All content successfully deleted, may be waiting on finalization} {Type:NamespaceContentRemaining Status:True LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:SomeResourcesRemain Message:Some resources are remaining: pods. has 4 resource instances} {Type:NamespaceFinalizersRemaining Status:False LastTransitionTime:2024-02-05 15:15:30 +0000 UTC Reason:ContentHasNoFinalizers Message:All content-preserving finalizers finished}]}
...
...
...

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.
=== Sonobuoy conformance tests passed in 104s ===
make[1]: Leaving directory '/home/ubuntu/src/github.com/SovereignCloudStack/k8s-cluster-api-provider/terraform'

Custom ENV config

use_ovn_lb_provider = "true"

@scs-zuul
Copy link

scs-zuul bot commented Feb 5, 2024

Build failed (e2e-test pipeline).
https://zuul.scs.community/t/SCS/buildset/0940976cc68848f0a9430cb9e0ae1c65

k8s-cluster-api-provider-e2e-conformance FAILURE in 25m 15s

Warning:

Custom ENV config

use_ovn_lb_provider = "true"

@scs-zuul
Copy link

scs-zuul bot commented Feb 6, 2024

Build succeeded (e2e-test pipeline).
https://zuul.scs.community/t/SCS/buildset/4e16dabd40194fde9b7c08fcdeaa24e0

✔️ k8s-cluster-api-provider-e2e-conformance SUCCESS in 2h 15m 31s

Warning:

Sonobouy results === Collecting results === time="2024-02-06T07:55:45Z" level=info msg="delete request issued" dry-run=false kind=namespace namespace=sonobuoy time="2024-02-06T07:55:45Z" level=info msg="delete request issued" dry-run=false kind=clusterrolebindings names="[sonobuoy-serviceaccount-sonobuoy]" time="2024-02-06T07:55:45Z" level=info msg="delete request issued" dry-run=false kind=clusterroles names="[sonobuoy-serviceaccount-sonobuoy]" time="2024-02-06T07:55:45Z" level=info msg="delete request issued" dry-run=false kind=namespace namespace=pods-3978 Plugin: e2e Status: passed Total: 7213 Passed: 382 Failed: 0 Skipped: 6831

Plugin: systemd-logs
Status: passed
Total: 4
Passed: 4
Failed: 0
Skipped: 0

Run Details:
API Server version: v1.27.8
Node health: 4/4 (100%)
Pods health: 33/34 (97%)
Details for failed pods:
pods-3978/pod-qos-class-a72dbb73-33b2-4e81-88e0-42cd2bac958c Ready:False: ContainersNotReady: containers with unready status: [agnhost]
Errors detected in files:
Errors:
1266 podlogs/kube-system/kube-controller-manager-pr687-76b45c-5p8gp-r8gjc/logs/kube-controller-manager.txt
834 podlogs/kube-system/snapshot-controller-55687d7977-sqkg5/logs/snapshot-controller.txt
217 podlogs/kube-system/cilium-zgvfw/logs/cilium-agent.txt
165 podlogs/kube-system/cilium-xvmrk/logs/cilium-agent.txt
141 podlogs/kube-system/kube-scheduler-pr687-76b45c-5p8gp-r8gjc/logs/kube-scheduler.txt
107 podlogs/kube-system/kube-apiserver-pr687-76b45c-5p8gp-r8gjc/logs/kube-apiserver.txt
106 podlogs/kube-system/cilium-zjdvp/logs/cilium-agent.txt
10 podlogs/kube-system/openstack-cloud-controller-manager-ct68f/logs/openstack-cloud-controller-manager.txt
8 podlogs/kube-system/cilium-smzh8/logs/cilium-agent.txt
8 podlogs/kube-system/kube-proxy-lflwx/logs/kube-proxy.txt
8 podlogs/kube-system/kube-proxy-f2nlr/logs/kube-proxy.txt
8 podlogs/kube-system/kube-proxy-8kmff/logs/kube-proxy.txt
8 podlogs/kube-system/kube-proxy-nfkww/logs/kube-proxy.txt
4 podlogs/kube-system/etcd-pr687-76b45c-5p8gp-r8gjc/logs/etcd.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-dlm9s/logs/node-driver-registrar.txt
1 podlogs/kube-system/cilium-operator-788c7d7585-ftzh5/logs/cilium-operator.txt
1 podlogs/sonobuoy/sonobuoy-e2e-job-c09fb3bfbab54093/logs/e2e.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-n8tkj/logs/node-driver-registrar.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-h7vdq/logs/node-driver-registrar.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-xw9v9/logs/node-driver-registrar.txt
Warnings:
2023 podlogs/kube-system/etcd-pr687-76b45c-5p8gp-r8gjc/logs/etcd.txt
540 podlogs/kube-system/kube-controller-manager-pr687-76b45c-5p8gp-r8gjc/logs/kube-controller-manager.txt
197 podlogs/kube-system/cilium-zgvfw/logs/cilium-agent.txt
153 podlogs/kube-system/cilium-xvmrk/logs/cilium-agent.txt
109 podlogs/kube-system/cilium-zjdvp/logs/cilium-agent.txt
102 podlogs/kube-system/kube-apiserver-pr687-76b45c-5p8gp-r8gjc/logs/kube-apiserver.txt
55 podlogs/kube-system/kube-scheduler-pr687-76b45c-5p8gp-r8gjc/logs/kube-scheduler.txt
14 podlogs/kube-system/openstack-cloud-controller-manager-ct68f/logs/openstack-cloud-controller-manager.txt
11 podlogs/kube-system/cilium-smzh8/logs/cilium-agent.txt
5 podlogs/kube-system/csi-cinder-nodeplugin-h7vdq/logs/node-driver-registrar.txt
4 podlogs/kube-system/csi-cinder-nodeplugin-n8tkj/logs/node-driver-registrar.txt
3 podlogs/kube-system/csi-cinder-controllerplugin-86884db55c-pshc9/logs/csi-attacher.txt
2 podlogs/kube-system/csi-cinder-controllerplugin-86884db55c-pshc9/logs/csi-provisioner.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-dlm9s/logs/node-driver-registrar.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-xw9v9/logs/liveness-probe.txt
2 podlogs/kube-system/csi-cinder-nodeplugin-xw9v9/logs/node-driver-registrar.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-dlm9s/logs/liveness-probe.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-n8tkj/logs/liveness-probe.txt
1 podlogs/kube-system/cilium-operator-788c7d7585-ftzh5/logs/cilium-operator.txt
1 podlogs/kube-system/metrics-server-5fb5598cf8-sdt7s/logs/metrics-server.txt
1 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt
1 podlogs/kube-system/csi-cinder-nodeplugin-h7vdq/logs/liveness-probe.txt
1 podlogs/sonobuoy/sonobuoy-e2e-job-c09fb3bfbab54093/logs/e2e.txt
time="2024-02-06T07:55:46Z" level=info msg="delete request issued" dry-run=false kind=namespace namespace=sonobuoy
time="2024-02-06T07:55:46Z" level=info msg="delete request issued" dry-run=false kind=clusterrolebindings names="[]"
time="2024-02-06T07:55:46Z" level=info msg="delete request issued" dry-run=false kind=clusterroles names="[]"
time="2024-02-06T07:55:46Z" level=info msg="delete request issued" dry-run=false kind=namespace namespace=pods-3978

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[{Type:NamespaceDeletionDiscoveryFailure Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ResourcesDiscovered Message:All resources successfully discovered} {Type:NamespaceDeletionGroupVersionParsingFailure Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ParsedGroupVersions Message:All legacy kube types successfully parsed} {Type:NamespaceDeletionContentFailure Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ContentDeleted Message:All content successfully deleted, may be waiting on finalization} {Type:NamespaceContentRemaining Status:True LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:SomeResourcesRemain Message:Some resources are remaining: pods. has 5 resource instances} {Type:NamespaceFinalizersRemaining Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ContentHasNoFinalizers Message:All content-preserving finalizers finished}]}

Namespace "sonobuoy" has status {Phase:Terminating Conditions:[{Type:NamespaceDeletionDiscoveryFailure Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ResourcesDiscovered Message:All resources successfully discovered} {Type:NamespaceDeletionGroupVersionParsingFailure Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ParsedGroupVersions Message:All legacy kube types successfully parsed} {Type:NamespaceDeletionContentFailure Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ContentDeleted Message:All content successfully deleted, may be waiting on finalization} {Type:NamespaceContentRemaining Status:True LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:SomeResourcesRemain Message:Some resources are remaining: pods. has 4 resource instances} {Type:NamespaceFinalizersRemaining Status:False LastTransitionTime:2024-02-06 07:55:51 +0000 UTC Reason:ContentHasNoFinalizers Message:All content-preserving finalizers finished}]}
...
...
...
...

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

Found 1 namespaces that still need to be deleted: [pods-3978]

Namespace "sonobuoy" has been deleted

Deleted all ClusterRoles and ClusterRoleBindings.

All E2E namespaces deleted
=== Sonobuoy conformance tests passed in 6556s ===
make[1]: Leaving directory '/home/ubuntu/src/github.com/SovereignCloudStack/k8s-cluster-api-provider/terraform'

Custom ENV config

use_ovn_lb_provider = "true"

Copy link
Member

@matofeder matofeder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM,

  • end-to-end tests prove that this works
  • now we could consider OVN LB as a default (but this could be addressed in a separate PR)

Copy link
Contributor

@DEiselt DEiselt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chess-knight chess-knight merged commit 47f12bb into main Feb 26, 2024
4 checks passed
@chess-knight chess-knight deleted the feat/ovn_lb branch February 26, 2024 07:54
@chess-knight
Copy link
Member Author

@garloff, can you please close kubernetes/cloud-provider-openstack#2234?

@garloff
Copy link
Member

garloff commented Mar 13, 2024

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix return value for handling ovn load balancer OCCM: Optionally support TCP Health-Mon for Loadbalancer
4 participants