Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP #203

Merged
merged 1 commit into from
Feb 4, 2024

Conversation

usernameisnull
Copy link
Contributor

@usernameisnull usernameisnull commented Dec 22, 2023

when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP
related #202 #159

What is the purpose of the change

when the broker uses hostNetwork, the broker's IP returned by NameServer is the IP of the k8s node where the broker is located.

This is because the broker uses the cluster ip(or other ip) instead of its own pod ip when registering.

The root cause is that when using hostnetwork, the host's network is used, including /etc/hosts. When generating the brokerIP1 configuration project, use the hostname -i command in the images/broker/alpine/brokerGenConfig.sh to obtain the ip.

/images/broker/alpine/brokerGenConfig.sh#L30

echo "brokerIP1=`hostname -i`" >> $BROKER_CONFIG_FILE

but in many cases this command does not correctly obtain the correct ip.
for exampels:

#/etc/hosts
127.0.0.1 localhost
127.0.1.1 master01

bash-4.4# hostname -i
127.0.1.1

when use 127.0.1.1 to register, the nameserver will select the IP based on its own algorithm. Most of the time it is not the status.podIP of the pod we want: https://github.com/apache/rocketmq/blob/2c898c9b31bf195174cf1e3a626a7c61f7576381/common/src/main/java/org/apache/rocketmq/common/utils/NetworkUtil.java#L91

#/etc/hosts
127.0.0.1 localhost
127.0.1.1 master01
10.6.178.178 master01

bash-4.4# hostname -i
127.0.1.1 10.6.178.178

check the log, it seems to be successful
image

but when we used the mqadmin command to check the registered broker address, we found that this was not the case.
image

Brief changelog

When setting up with hostNetwork option, we explicitly add an environment variable HOST_NETWORK_POD_IP on the pod. When obtaining brokerIP1, we obtain it from this environment variable and register it with the nameserver.

Verifying this change

the log from the pod:
image

the podIP:
image

image

use mqadmin see the broker address:
image

Please go through this checklist to help us incorporate your contribution quickly and easily.

Notice: It would be helpful if you could finish the following checklist (the last one is not necessary) before request the community to review your PR.

  • Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Check RBAC rights for Kubernetes roles.
  • Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist.
  • Run make docker-build to build docker image for operator, try your changes from Pod inside your Kubernetes cluster, not just locally. Also provide screenshots to show that the RocketMQ cluster is healthy after the changes.
  • Before committing your changes, remember to run make manifests to make sure the CRD files are updated.
  • Update documentation if necessary.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

@usernameisnull
Copy link
Contributor Author

PTAL @caigy

@shangjin92
Copy link
Contributor

BROKER_NETWORK_DEV=`/sbin/route |awk '/^default/{print $NF}'`
BROKER_IP=`ip a s ${BROKER_NETWORK_DEV} |awk -F' +|/' '/ inet /{print $3}'`
echo "brokerIP1=${BROKER_IP}" >> $BROKER_CONFIG_FILE

You can try this instead, that doesn't require manual IP specification, it works for me.

@usernameisnull
Copy link
Contributor Author

BROKER_NETWORK_DEV=`/sbin/route |awk '/^default/{print $NF}'`
BROKER_IP=`ip a s ${BROKER_NETWORK_DEV} |awk -F' +|/' '/ inet /{print $3}'`
echo "brokerIP1=${BROKER_IP}" >> $BROKER_CONFIG_FILE

You can try this instead, that doesn't require manual IP specification, it works for me.

thank you, I will try this

Copy link
Contributor

@caigy caigy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO status.podIP can achieve it, so are there any reasons to get the pod ip by a complicated command?

images/broker/alpine/brokerGenConfig.sh Outdated Show resolved Hide resolved
@usernameisnull
Copy link
Contributor Author

PTAL @caigy

Copy link
Contributor

@caigy caigy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@caigy caigy merged commit 455dc24 into apache:master Feb 4, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants