add proposal for Locality LoadBalance #574

derekwin · 2024-07-15T06:53:14Z

What type of PR is this?
/kind enhancement

What this PR does / why we need it:
add proposal for Locality LB

kmesh-bot · 2024-07-15T06:53:25Z

Welcome @derekwin! It looks like this is your first PR to kmesh-net/kmesh 🎉

LiZhenCheng9527 · 2024-07-15T07:47:20Z

Would you like to share your issue at Thursday's community meeting?

LiZhenCheng9527 · 2024-07-15T07:56:05Z

docs/proposal/kmesh_support_localityLB.md

+
+### Motivation
+
+Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.


Suggested change

Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.

Currently, Kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.

Unified capitalisation of initial letters in Kmesh

LiZhenCheng9527 · 2024-07-15T07:58:11Z

docs/proposal/kmesh_support_localityLB.md

+
+#### case 1. locality failover
+1. Destination Rule
+Same as Istion. Parse rules specify configuration for Locality load balancing. (todo: outlier detection settings to detect and evict unhealthy hosts from the load balancing pool.)


what is istion? Istio?

hzxuzhonghu · 2024-07-16T01:40:50Z

/ok-to-test

codecov · 2024-07-16T01:53:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.80%. Comparing base (433592b) to head (b2caa7c).
Report is 217 commits behind head on main.

see 29 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dda7049...b2caa7c. Read the comment docs.

hzxuzhonghu · 2024-07-16T02:35:49Z

docs/proposal/kmesh_support_localityLB.md

+
+Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.
+
+#### Goals


Suggested change

#### Goals

### Goals

hzxuzhonghu · 2024-07-16T02:37:16Z

docs/proposal/kmesh_support_localityLB.md

+
+1. prioritize add locality load balancing capabilities in the workload mode.
+
+2. two types of locality load balancing : locality failover, locality weighted distribution.


I am not sure how locality weighted distribution can be implemented in workload mode. The workload api does not support weight actually

hzxuzhonghu · 2024-07-16T02:41:49Z

docs/proposal/kmesh_support_localityLB.md

+#### case 1. locality failover
+1. Destination Rule
+Same as Istion. Parse rules specify configuration for Locality load balancing. (todo: outlier detection settings to detect and evict unhealthy hosts from the load balancing pool.)
+    - Outlier detection should occur before load balancing.


This does not suite worklaod mode as workload api does not include outlier setting. It do LB based on where the endpoint resides.

derekwin · 2024-07-16T03:45:39Z

Would you like to share your issue at Thursday's community meeting?

yes

derekwin · 2024-07-25T05:53:55Z

I have updated the proposal.

derekwin · 2024-07-26T11:58:02Z

Propose a new implementation for a location matching algorithm that avoids circular computations while also reducing the amount of data needed to be stored in BPF maps. detail: https://github.com/derekwin/treemap/tree/master
Welcome to offer suggestions to further improve the approach.

Okabe-Rintarou-0 · 2024-07-30T09:24:02Z

if no conflict, there is no need to merge main branch.
If there are some conflicts, to get a clearer commit history, you should:

git rebase main

then fix some conflicts, and then

git rebase --continue
git push --force

the DCO github action failed, because it asks you to commit with your signature, which can be attached with -s flag:

git commit -s -m 'something to say'

hzxuzhonghu

I wish to see more api design instead of function implement in the proposal

How do you express the priority level, and how do you match the client locality with the endpoints

hzxuzhonghu · 2024-08-02T09:03:00Z

docs/proposal/kmesh_support_localityLB.md

+
+1. prioritize add locality load balancing capabilities in the workload mode.
+
+2. locality load balancing mode: locality failover.


how about strict mode

hzxuzhonghu · 2024-08-02T09:06:06Z

docs/proposal/kmesh_support_localityLB.md

+```
+https://pkg.go.dev/istio.io/istio/pkg/workloadapi#LoadBalancing_Scope
+
+2. calculate locality match rank


group endpoints with prority

hzxuzhonghu · 2024-08-02T09:07:20Z

docs/proposal/kmesh_support_localityLB.md

+
+3. choose endpoint
+
+Randomly select one endpoint from the group with the highest rank as the service backend.


Suggested change

Randomly select one endpoint from the group with the highest rank as the service backend.

Randomly select one endpoint from the group with the highest priority

And add more comments what we do if all the endpoints of high priority is unhealthy

And for the strict mode, how would you select the endpoint, i would like to see that

hzxuzhonghu · 2024-08-02T09:08:43Z

docs/proposal/kmesh_support_localityLB.md

+
+4. maybe more？ Panic threshold
+
+When the proportion of healthy endpoints in the high-rank group falls below the panic threshold, select endpoints from the next rank group.


I donot care about this at first. First respect workload healthy status

hzxuzhonghu · 2024-08-02T09:09:34Z

docs/proposal/kmesh_support_localityLB.md

+    __u32 waypoint_addr;
+    __u32 waypoint_port;
+    // 增加健康状态 healthStatus
+    // 增加locality信息


Please supplement what does this field look like.

hzxuzhonghu · 2024-08-12T01:41:49Z

docs/proposal/kmesh_support_localityLB.md

+Add corresponding fields to the `pkg/controller/workload/bpfcache/service.go`, and update logic to `pkg/controller/workload/workload_processor.go`
+
+2. Configure the locality (region, zone, subzone) and health status （HEALTHY, UNHEALTHY）of the backend. This corresponds to the message in workload.proto.
+> Although the current workload API defines seven scopes, when configuring a pod's locality, only region, zone, and subzone are configured. Therefore, matching capabilities can only be realized for these three scopes.


only region, zone, and subzone are configured

where do you get this conclusion, at least NODE is supported now

I misunderstood it before. I saw that in ztunnel, the NODE, NETWORK, and CLUSTER information were maintained within the workload, and I considering adding these informations to the bpf map of the backend later.

hzxuzhonghu · 2024-08-12T01:45:29Z

docs/proposal/kmesh_support_localityLB.md

+    __u32 service[MAX_SERVICE_COUNT];
+    struct ip_addr wp_addr;
+    __u32 waypoint_port;
+    __u8 health_status; // workload_health_status_t: HEALTHY, UNHEALTHY


Currently we filtered out unhealthy workload

So, only healthy workloads are stored in bpf map by the control plane. Does locality load balance not need to concern whether the workload is healthy or not?

We can make it simpler, even the priority set can be calculated in user space.

Priority computation between localities occurs when a new flow is came. If priority calculation takes place at the control plane, my understanding is that we would need to precompute all possible scenarios（We are unable to perform event-driven programming that interoperates with user space, right?）, then hash different situations and store them in a BPF map. The kernel space would then query the map using source and destination locality information to obtain priority information. To simplify the problem, we could arrange combinations based on the specific values pointed to by the six routing options in scope (also including cases where only some of these match). This approach has two potential issues:
Firstly, userspace must enumerate all possible scenarios, which becomes particularly burdensome as the richness of locality information increases, leading to an exponential growth in the number of situations to be stored. Secondly, the BPF map would have to store all aforementioned scenarios, with each scenario existing in the form of the prio_map as currently designed.

My concern mainly on : 1. ebpf instruction limit 2. dataplane sorting performance Worth a try though

hzxuzhonghu · 2024-08-12T01:47:52Z

docs/proposal/kmesh_support_localityLB.md

+    struct ip_addr wp_addr;
+    __u32 waypoint_port;
+    __u8 health_status; // workload_health_status_t: HEALTHY, UNHEALTHY
+    locality_t locality;


What is locality_t then?

I will add it in next commit.

derekwin · 2024-09-07T09:30:06Z

new proposal of locality LB in user-space logic

hzxuzhonghu · 2024-09-10T01:49:03Z

docs/proposal/kmesh_support_localityLB.md

+typedef struct {
+    __u32 service_id; // service id
+    __u32 rank; // rank
+} prio_key;


what is the relationship with endpoint_map?

When we use this map, and when we use the other?

hzxuzhonghu · 2024-09-10T01:50:54Z

docs/proposal/kmesh_support_localityLB.md

+} prio_key;
+typedef struct {
+    __u32 count; // count of current prio
+    __u32 uid_list[MAP_SIZE_OF_PRIO]; // workload_uid to backend


This can waste memory

So why not add priority to endpoint key

update endpoint_key {

typedef struct {
__u32 service_id; // service id
__u32 Priority,
__u32 backend_index; // if endpoint_count = 3, then backend_index = 0/1/2
} endpoint_key;

derekwin · 2024-09-28T03:16:19Z

New design has been updated to the proposal and the correspond code Pr is here #900

hzxuzhonghu · 2024-09-29T03:00:31Z

docs/proposal/kmesh_support_localityLB.md

+```
+typedef struct {
+    __u32 service_id;    // service id
+    __u32 prio;          // prio means rank, 6 means match all, and 0 means match nothing


By adding this, how do we select a endpoint now?

For random lb mode，workload will only be added to endpoint with maxprio(6).
For locality lb mode，workload will be added to endpoint with rank that calculated by matching kmesh processor's locality info with workload's locality info.
We also record the count number of endpoints belongs to which prio in serviceValue, so that we can use it as it before.
In the bpf prog, If service is in random lb mode, we can search endpoint with maxprio. if it is in locality lb mode, we will iter prio from maxprio to 0, if count of that prio >0, which means there have one or more endpoints in that prio, we can choose one workload index by random int value with count, and get endpoint whith serviceId, prio and workloadIndex.

make sense, not the bpf map update should be a little bit tricky

0 means nothing means what?

prio value is from 0 to 6, 0 means the lowest priolity.

hmm, i would suggest the opposite. Because we can search from the highest priority easily

Ok, I have updated it.

hzxuzhonghu · 2024-09-29T06:31:12Z

docs/proposal/kmesh_support_localityLB.md

+
+workload.h
+```
+#define MAX_PRIO                  6


IIUC the max priority rank can be 7

with
// Prefer traffic in the same region.
LoadBalancing_REGION LoadBalancing_Scope = 1
// Prefer traffic in the same zone.
LoadBalancing_ZONE LoadBalancing_Scope = 2
// Prefer traffic in the same subzone.
LoadBalancing_SUBZONE LoadBalancing_Scope = 3
// Prefer traffic on the same node.
LoadBalancing_NODE LoadBalancing_Scope = 4
// Prefer traffic in the same cluster.
LoadBalancing_CLUSTER LoadBalancing_Scope = 5
// Prefer traffic in the same network.
LoadBalancing_NETWORK LoadBalancing_Scope = 6

prio value is from 0 to 6, so i set MAX_PRIO as 6, which actually is the 7th rank.

Signed-off-by: seclee <[email protected]> Signed-off-by: derekwin <[email protected]>

Signed-off-by: derekwin <[email protected]>

docs/proposal/kmesh_support_localityLB.md

kmesh-bot · 2024-10-11T02:12:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hzxuzhonghu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kmesh-bot added the kind/enhancement New feature or request label Jul 15, 2024

kmesh-bot requested review from bitcoffeeiux and kevin-wangzefeng July 15, 2024 06:53

kmesh-bot added size/M do-not-merge/contains-merge-commits labels Jul 15, 2024

LiZhenCheng9527 reviewed Jul 15, 2024

View reviewed changes

kmesh-bot added the ok-to-test label Jul 16, 2024

hzxuzhonghu reviewed Jul 16, 2024

View reviewed changes

kmesh-bot added size/L and removed size/M labels Jul 25, 2024

hzxuzhonghu reviewed Aug 2, 2024

View reviewed changes

derekwin force-pushed the main branch from 48a0b47 to a19444b Compare August 8, 2024 12:41

kmesh-bot removed the do-not-merge/contains-merge-commits label Aug 8, 2024

derekwin deleted the branch kmesh-net:main August 8, 2024 12:47

derekwin closed this Aug 8, 2024

derekwin deleted the main branch August 8, 2024 12:47

derekwin restored the main branch August 8, 2024 12:49

derekwin reopened this Aug 8, 2024

derekwin force-pushed the main branch from a19444b to b6583c6 Compare August 10, 2024 10:57

derekwin requested review from hzxuzhonghu and LiZhenCheng9527 August 10, 2024 23:13

hzxuzhonghu reviewed Aug 12, 2024

View reviewed changes

derekwin force-pushed the main branch from b6583c6 to b2caa7c Compare September 7, 2024 09:26

kmesh-bot added size/XL and removed size/L labels Sep 7, 2024

hzxuzhonghu reviewed Sep 10, 2024

View reviewed changes

kmesh-bot added size/XXL and removed size/XL labels Sep 26, 2024

derekwin force-pushed the main branch from 7ab43b6 to 85feec7 Compare September 28, 2024 03:13

derekwin force-pushed the main branch 2 times, most recently from dbf6e1d to 3623f1e Compare September 28, 2024 09:16

hzxuzhonghu reviewed Sep 29, 2024

View reviewed changes

derekwin and others added 5 commits October 11, 2024 08:36

add proposal for Locality LB

1aad4eb

Signed-off-by: seclee <[email protected]> Signed-off-by: derekwin <[email protected]>

Revise the proposal based on the recommendations.

80f368b

Signed-off-by: seclee <[email protected]> Signed-off-by: derekwin <[email protected]>

locality LB logic in user-space

3fbaa0f

Signed-off-by: derekwin <[email protected]>

new proposal for locality loadbalance

caa67d0

Signed-off-by: derekwin <[email protected]>

change prio from 0 to 6

752551a

Signed-off-by: derekwin <[email protected]>

derekwin force-pushed the main branch from a96f217 to 752551a Compare October 11, 2024 00:37

hzxuzhonghu approved these changes Oct 11, 2024

View reviewed changes

docs/proposal/kmesh_support_localityLB.md Show resolved Hide resolved

kmesh-bot added the approved label Oct 11, 2024


		### Motivation

		Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.


		Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.

		#### Goals


		1. prioritize add locality load balancing capabilities in the workload mode.

		2. two types of locality load balancing : locality failover, locality weighted distribution.


		1. prioritize add locality load balancing capabilities in the workload mode.

		2. locality load balancing mode: locality failover.


		3. choose endpoint

		Randomly select one endpoint from the group with the highest rank as the service backend.


		4. maybe more？ Panic threshold

		When the proportion of healthy endpoints in the high-rank group falls below the panic threshold, select endpoints from the next rank group.

add proposal for Locality LoadBalance #574

Are you sure you want to change the base?

add proposal for Locality LoadBalance #574

Conversation

derekwin commented Jul 15, 2024

kmesh-bot commented Jul 15, 2024

LiZhenCheng9527 commented Jul 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hzxuzhonghu commented Jul 16, 2024

codecov bot commented Jul 16, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derekwin commented Jul 16, 2024

derekwin commented Jul 25, 2024

derekwin commented Jul 26, 2024

Okabe-Rintarou-0 commented Jul 30, 2024 • edited Loading

hzxuzhonghu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derekwin commented Sep 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derekwin commented Sep 28, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kmesh-bot commented Oct 11, 2024

codecov bot commented Jul 16, 2024 •

edited

Loading

Okabe-Rintarou-0 commented Jul 30, 2024 •

edited

Loading