IP address of VIP is not reported

[jcpowermac]

Describe the bug

Suppose the following openvswitch bridge

5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:50:56:90:c5:eb brd ff:ff:ff:ff:ff:ff
    inet 10.93.60.245/25 brd 10.93.60.255 scope global dynamic noprefixroute br-ex
       valid_lft 2173sec preferred_lft 2173sec
    inet 169.254.0.2/17 brd 169.254.127.255 scope global br-ex
       valid_lft forever preferred_lft forever
    inet 10.93.60.131/32 scope global vip
       valid_lft forever preferred_lft forever
    inet6 fd65:a1a8:60ad:981::13f/128 scope global dynamic noprefixroute 
       valid_lft 39977sec preferred_lft 23777sec
    inet6 fd65:a1a8:60ad:981:3a94:5abb:c111:19fd/64 scope global dynamic noprefixroute 
       valid_lft 2591801sec preferred_lft 604601sec
    inet6 fe80::89c2:3c12:3d2b:ee0e/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

All addresses except 10.93.60.131/32 are reported via tools to vCenter,

Reproduction steps

1. Recent version (>=4.14) of RH OpenShift or OKD, api or ingress vip are not reported in tools

Expected behavior

VIP reported via tools

Additional context

[root@jcallen3-mvbqq-worker-0-49qbs /]# vmtoolsd -v
VMware Tools daemon, version 12.4.0.48309 (build-23259341)
[root@jcallen3-mvbqq-worker-0-49qbs /]#

[PaTHml]

Thanks for reporting the issue. Filed an internal bug for the same.

We do need more information on the setup.

• ESXi version (or Workstation, or Fusion versions and host operating system version)
• vCenter version
• Virtual machine guest operating system and version.

Any setup details for reproduction?

Since this seems to deal with a Kubernetes container, we'll need details on that setup as well.

[songyu555]

Hello,

Providing the information on behalf of RH team..

ESXi version: VMware ESXi hypervisor version, 7.0.3, 21686933
vCenter version: 7.03u3
Virtual machine guest operating system and version: CoreOS 4.14 / Openshift v4.14
NSX-T version: NSX version 3.2

DHCP service is enabled on NSX-T to provide dynamic IPs for VMs.
ARP snooping is the default , (limited 1)

When issue occurs, we are doing Upgrade from Openshift 4.12.56 to Openshift 4.14.33 version ( 3x Master, 3x infra and number of workers).

Thanks,
Yu Song

[jcpowermac]

@PaTHml

vCenter: 8.0.2, 23504390
ESXi: 8.0.2, 23305546

I tested with OKD so that anyone can install without having to deal with the pull secret.

https://docs.okd.io/4.16/installing/installing_vsphere/ipi/installing-vsphere-installer-provisioned-customizations.html#installation-installer-provisioned-vsphere-config-yaml_installing-vsphere-installer-provisioned-customizations

Here is an example of an install-config.yaml

apiVersion: v1
baseDomain: vmc.devcluster.openshift.com
metadata:
  name: jcallen3
networking:
  machineNetwork:
  - cidr: 10.93.60.0/24
platform:
  vsphere:
    apiVIP: 10.93.60.130
    ingressVIP: 10.93.60.131
    vcenters:
    - server: vcenter.ci.ibmc.devcluster.openshift.com
      user: [email protected]
      password: ''
      datacenters:
      - cidatacenter
    failureDomains:
    - name: us-east-1
      region: us-east
      zone: us-east-1a
      server: vcenter.ci.ibmc.devcluster.openshift.com
      topology:
        datacenter: cidatacenter
        computeCluster: /cidatacenter/host/cicluster
        networks:
        - ci-vlan-981-testing 
        datastore: /cidatacenter/datastore/vsanDatastore
pullSecret: |
  {"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}}

sshKey: 

You will need to create two A records that correspond to the apiVIP and ingressVIP

➜  vsphere-ipi host -t A foo.apps.jcallen3.vmc.devcluster.openshift.com
foo.apps.jcallen3.vmc.devcluster.openshift.com has address 10.93.60.131
➜  vsphere-ipi host -t A api.jcallen3.vmc.devcluster.openshift.com    
api.jcallen3.vmc.devcluster.openshift.com has address 10.93.60.130

Running the install...

oc adm release extract --tools registry.ci.openshift.org/origin/release-scos:4.17.0-0.okd-scos-2024-08-29-100725

export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE="registry.ci.openshift.org/origin/release-scos:4.17.0-0.okd-scos-2024-08-29-100725"

openshift-install create cluster

The OS is CentOS 9 based when using OKD but this occurs in RHCOS which is RHEL based for multiple releases through RHEL 9 to 9.4.

We currently only use dvs not nsx-t.