Skip to content

Commit

Permalink
Merge branch 'main' into fix-cicd-k3s-calico
Browse files Browse the repository at this point in the history
  • Loading branch information
UltraInstinct14 authored Oct 7, 2024
2 parents e360692 + f8cb36b commit 912e864
Show file tree
Hide file tree
Showing 190 changed files with 7,895 additions and 742 deletions.
11 changes: 3 additions & 8 deletions .github/workflows/k3s-base-sanity.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,17 +38,12 @@ jobs:
- uses: actions/checkout@v2
with:
submodules: recursive
- run: curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --disable traefik --disable servicelb --disable-cloud-controller --kubelet-arg cloud-provider=external" K3S_KUBECONFIG_MODE="644" sh -
- run: curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --disable traefik --disable servicelb" K3S_KUBECONFIG_MODE="644" sh -
- run: |
sleep 10
kubectl "${{ env.KUBECONFIG }}" taint nodes --all node.cloudprovider.kubernetes.io/uninitialized=false:NoSchedule-
sleep 60
sleep 80
kubectl "${{ env.KUBECONFIG }}" get nodes
kubectl "${{ env.KUBECONFIG }}" get pods -A
wget https://github.com/loxilb-io/loxi-ccm/raw/master/manifests/loxi-ccm-k3s.yaml
kubectl "${{ env.KUBECONFIG }}" apply -f ./loxi-ccm-k3s.yaml
sleep 60
kubectl "${{ env.KUBECONFIG }}" get pods -A
sleep 20
- run: |
cd cicd/k3s-base-sanity/
./config.sh "${{ env.KUBECONFIG }}"
Expand Down
36 changes: 36 additions & 0 deletions .github/workflows/k8s-calico-ipsec-ha.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: K8s-Calico-Cluster-IPSec-HA-Sanity-CI
on:
schedule:
# Runs "At 19:00 UTC every day-of-week"
- cron: '0 19 * * *'
workflow_dispatch:
inputs:
testName:
description: 'Test Run-Name'
required: true
default: 'k8s-calico-cluster-ipsec-ha'
jobs:
test-runner:
name: k8s-calico-cluster-ipsec-ha-sanity
runs-on: [self-hosted, large]
if: github.repository == 'loxilb-io/loxilb'
&& github.event.inputs.tagName == ''
steps:
- name: Checkout
uses: actions/checkout@v2
with:
submodules: recursive

- name: Run the test
run: |
cd cicd/k8s-calico-ipsec-ha
./config.sh
./validation_with_sctp.sh
cd -
- name: Clean test-bed
if: success() || failure()
run: |
cd cicd/k8s-calico-ipsec-ha || true
./rmconfig.sh
cd -
36 changes: 36 additions & 0 deletions .github/workflows/k8s-flannel-incluster-multus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: K8s-Flannel-Incluster-Multus-Sanity-CI
on:
# schedule:
# Runs "At 11:00 UTC every day-of-week"
#- cron: '0 11 * * *'
workflow_dispatch:
inputs:
testName:
description: 'Test Run-Name'
required: true
default: 'k8s-flannel-incluster-multus'
jobs:
test-runner:
name: k8s-flannel-incluster-multus-sanity
runs-on: [self-hosted, large]
if: github.repository == 'loxilb-io/loxilb'
&& github.event.inputs.tagName == ''
steps:
- name: Checkout
uses: actions/checkout@v2
with:
submodules: recursive

- name: Run the test
run: |
cd cicd/k8s-flannel-incluster-multus
./config.sh
./validation.sh
cd -
- name: Clean test-bed
if: success() || failure()
run: |
cd cicd/k8s-flannel-incluster-multus || true
./rmconfig.sh
cd -
36 changes: 36 additions & 0 deletions .github/workflows/sctpmh-seagull-sanity.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: SCTP-MH-LB-Seagull-Sanity-CI
on:
# schedule:
# Runs "At 11:00 UTC every day-of-week"
#- cron: '0 11 * * *'
workflow_dispatch:
inputs:
testName:
description: 'Test Run-Name'
required: true
default: 'sctpmh-seagull-sanity'
jobs:
test-runner:
name: k8s-calico-incluster-sanity
runs-on: [self-hosted, large]
if: github.repository == 'loxilb-io/loxilb'
&& github.event.inputs.tagName == ''
steps:
- name: Checkout
uses: actions/checkout@v2
with:
submodules: recursive

- name: Run the test
run: |
cd cicd/sctpmh-seagull
./config.sh
./validation.sh
cd -
- name: Clean test-bed
if: success() || failure()
run: |
cd cicd/sctpmh-seagull || true
./rmconfig.sh
cd -
2 changes: 1 addition & 1 deletion README-KOR.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ loxilb는 기본적으로 L4 로드 밸런서/서비스 프록시로 작동합
- [How-To : ingress-nginx와 함께 loxilb 배포](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/loxilb-nginx-ingress.md)

## 배경 지식
- [eBPF란 무엇인가](ebpf.md)
- [eBPF란 무엇인가](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/ebpf.md)
- [k8s 서비스 - 로드 밸런서란 무엇인가](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/lb.md)
- [간단한 아키텍처](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/arch.md)
- [코드 조직](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/code.md)
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,17 +79,20 @@ Telco-cloud requires load-balancing and communication across various interfaces/

## Getting Started
#### loxilb as ext-cluster pod
- [K8s : loxilb ext-mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k8s-flannel-ext.md)
- [K3s : loxilb with default flannel](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k3s_quick_start_flannel.md)
- [K3s : loxilb with calico](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k3s_quick_start_calico.md)
- [K3s : loxilb with cilium](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/quick_start_with_cilium.md)
- [K0s : loxilb with default kube-router networking](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k0s_quick_start.md)
- [EKS : loxilb ext-mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/eks-external.md)

#### loxilb as in-cluster pod
- [K8s : loxilb in-cluster mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k8s-flannel-incluster.md)
- [K3s : loxilb in-cluster mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k3s_quick_start_incluster.md)
- [K0s : loxilb in-cluster mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/k0s_quick_start_incluster.md)
- [MicroK8s : loxilb in-cluster mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/microk8s_quick_start_incluster.md)
- [EKS : loxilb in-cluster mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/eks-incluster.md)
- [RedHat OCP : loxilb in-cluster mode](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/rhocp-quickstart-incluster.md)

#### loxilb as service-proxy (kube-proxy replacement)
- [K3s : loxilb service-proxy with flannel](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/service-proxy-flannel.md)
Expand All @@ -108,9 +111,10 @@ Telco-cloud requires load-balancing and communication across various interfaces/
- [How-To : Deploy loxilb with multi-AZ HA support in AWS](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/aws-multi-az.md)
- [How-To : Deploy loxilb with multi-cloud HA support](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/multi-cloud-ha.md)
- [How-To : Deploy loxilb with ingress-nginx](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/loxilb-nginx-ingress.md)
- [How-To : Run loxilb in-cluster with secondary networks](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/loxilb-incluster-multus.md)

## Knowledge-Base
- [What is eBPF](ebpf.md)
- [What is eBPF](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/ebpf.md)
- [What is k8s service - load-balancer](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/lb.md)
- [Architecture in brief](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/arch.md)
- [Code organization](https://github.com/loxilb-io/loxilbdocs/blob/main/docs/code.md)
Expand Down
28 changes: 13 additions & 15 deletions api/loxinlp/nlp.go
Original file line number Diff line number Diff line change
Expand Up @@ -907,8 +907,9 @@ func ModLink(link nlp.Link, add bool) int {
Link: linkState, State: state, Mtu: mtu, Master: master, Real: real,
TunID: tunId, TunDst: tunDst, TunSrc: tunSrc})
if err != nil {
tk.LogIt(tk.LogError, "[NLP] Port %v, %v, %v, %v add failed\n", name, ifMac, state, mtu)
fmt.Println(err)
if !strings.Contains(err.Error(), "port exists") {
tk.LogIt(tk.LogError, "[NLP] Port %v, %v, %v, %v add failed\n", name, ifMac, state, mtu)
}
} else {
tk.LogIt(tk.LogInfo, "[NLP] Port %v, %v, %v, %v add [OK]\n", name, ifMac, state, mtu)
}
Expand All @@ -917,7 +918,6 @@ func ModLink(link nlp.Link, add bool) int {
ret, err = hooks.NetPortDel(&cmn.PortMod{Dev: name, Ptype: pType})
if err != nil {
tk.LogIt(tk.LogError, "[NLP] Port %v, %v, %v, %v delete failed\n", name, ifMac, state, mtu)
fmt.Println(err)
} else {
tk.LogIt(tk.LogInfo, "[NLP] Port %v, %v, %v, %v delete [OK]\n", name, ifMac, state, mtu)
}
Expand Down Expand Up @@ -1627,50 +1627,48 @@ func LbSessionGet(done bool) int {

if _, err := os.Stat(opt.Opts.ConfigPath + "/EPconfig.txt"); errors.Is(err, os.ErrNotExist) {
if err != nil {
tk.LogIt(tk.LogInfo, "[NLP] No EndPoint config file : %s \n", err.Error())
tk.LogIt(tk.LogInfo, "[NLP] Continuing without EP config file: %s\n", err.Error())
}
} else {
applyEPConfig()
}
tk.LogIt(tk.LogInfo, "[NLP] EndPoint done\n")
tk.LogIt(tk.LogInfo, "[NLP] EndPoint config process done\n")

if _, err := os.Stat(opt.Opts.ConfigPath + "/lbconfig.txt"); errors.Is(err, os.ErrNotExist) {
if err != nil {
tk.LogIt(tk.LogInfo, "[NLP] No load balancer config file : %s \n", err.Error())
tk.LogIt(tk.LogInfo, "[NLP] Continuing without LB config file : %s \n", err.Error())
}
} else {
applyLoadBalancerConfig()
}
tk.LogIt(tk.LogInfo, "[NLP] LoadBalancer config done\n")

tk.LogIt(tk.LogInfo, "[NLP] LoadBalancer done\n")
if _, err := os.Stat(opt.Opts.ConfigPath + "/sessionconfig.txt"); errors.Is(err, os.ErrNotExist) {
if err != nil {
tk.LogIt(tk.LogInfo, "[NLP] No Session config file : %s \n", err.Error())
tk.LogIt(tk.LogInfo, "[NLP] Continuing without Session config file : %s \n", err.Error())
}
} else {
applySessionConfig()
}
tk.LogIt(tk.LogInfo, "[NLP] Session config done\n")

tk.LogIt(tk.LogInfo, "[NLP] Session done\n")
if _, err := os.Stat(opt.Opts.ConfigPath + "/sessionulclconfig.txt"); errors.Is(err, os.ErrNotExist) {
if err != nil {
tk.LogIt(tk.LogInfo, "[NLP] No UlCl config file : %s \n", err.Error())
tk.LogIt(tk.LogInfo, "[NLP] Continuing without UlCl config file : %s \n", err.Error())
}
} else {
applyUlClConfig()
}
tk.LogIt(tk.LogInfo, "[NLP] Session UlCl config done\n")

tk.LogIt(tk.LogInfo, "[NLP] Session UlCl done\n")
if _, err := os.Stat(opt.Opts.ConfigPath + "/FWconfig.txt"); errors.Is(err, os.ErrNotExist) {
if err != nil {
tk.LogIt(tk.LogInfo, "[NLP] No Firewall config file : %s \n", err.Error())
tk.LogIt(tk.LogInfo, "[NLP] Continuing without Firewall config file : %s \n", err.Error())
}
} else {
applyFWConfig()
}
tk.LogIt(tk.LogInfo, "[NLP] Firewall done\n")

tk.LogIt(tk.LogInfo, "[NLP] LbSessionGet done\n")
tk.LogIt(tk.LogInfo, "[NLP] Firewall config done\n")
}

return 0
Expand Down
2 changes: 1 addition & 1 deletion cicd/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -561,7 +561,7 @@ function create_lb_rule() {
return
fi

hook=$($dexec llb1 tc filter show dev eth0 ingress | grep tc_packet_func)
hook=$($dexec $1 tc filter show dev eth0 ingress | grep tc_packet_func)
if [[ $hook != *"tc_packet_func"* ]]; then
echo "ERROR : No hook point found";
exit 1
Expand Down
3 changes: 2 additions & 1 deletion cicd/docker-k0s-lb/kube-loxilb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ rules:
resources:
- endpoints
- services
- namespaces
- services/status
verbs:
- get
Expand Down Expand Up @@ -111,7 +112,7 @@ spec:
- /bin/kube-loxilb
args:
- --loxiURL=http://172.17.0.2:11111
- --externalCIDR=192.168.82.100/32
- --cidrPools=defaultPool=192.168.82.100/32
#- --monitor
#- --setBGP
#- --setLBMode=1
Expand Down
2 changes: 1 addition & 1 deletion cicd/docker-k3s-calico/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ function create_lb_rule() {
echo "$1: loxicmd create lb ${args[*]}"
$dexec $1 loxicmd create lb ${args[*]}

hook=$($dexec llb1 tc filter show dev eth0 ingress | grep tc_packet_func)
hook=$($dexec $1 tc filter show dev eth0 ingress | grep tc_packet_func)
if [[ $hook != *"tc_packet_func"* ]]; then
echo "ERROR : No hook point found";
exit 1
Expand Down
3 changes: 2 additions & 1 deletion cicd/docker-k3s-calico/kube-loxilb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ rules:
resources:
- endpoints
- services
- namespaces
- services/status
verbs:
- get
Expand Down Expand Up @@ -111,7 +112,7 @@ spec:
- /bin/kube-loxilb
args:
- --loxiURL=http://172.17.0.2:11111
- --externalCIDR=192.168.163.247/32
- --cidrPools=defaultPool=192.168.163.247/32
#- --monitor
#- --setBGP
#- --setLBMode=1
Expand Down
2 changes: 1 addition & 1 deletion cicd/docker-k3s-cilium/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ function create_lb_rule() {
echo "$1: loxicmd create lb ${args[*]}"
$dexec $1 loxicmd create lb ${args[*]}

hook=$($dexec llb1 tc filter show dev eth0 ingress | grep tc_packet_func)
hook=$($dexec $1 tc filter show dev eth0 ingress | grep tc_packet_func)
if [[ $hook != *"tc_packet_func"* ]]; then
echo "ERROR : No hook point found";
exit 1
Expand Down
3 changes: 2 additions & 1 deletion cicd/docker-k3s-cilium/kube-loxilb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ rules:
resources:
- endpoints
- services
- namespaces
- services/status
verbs:
- get
Expand Down Expand Up @@ -111,7 +112,7 @@ spec:
- /bin/kube-loxilb
args:
- --loxiURL=http://172.17.0.2:11111
- --externalCIDR=192.168.163.247/32
- --cidrPools=defaultPool=192.168.163.247/32
#- --monitor
#- --setBGP
#- --setLBMode=1
Expand Down
4 changes: 2 additions & 2 deletions cicd/eks/kube-loxilb.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ rules:
resources:
- endpoints
- services
- namespaces
- services/status
verbs:
- get
Expand Down Expand Up @@ -111,8 +112,7 @@ spec:
- /bin/kube-loxilb
args:
- --loxiURL=http://$LOXILB_PRIVATE_IP:11111
- --externalCIDR=$LOXILB_PRIVATE_IP/32
#- --externalSecondaryCIDRs=124.124.124.1/24,125.125.125.1/24
- --cidrPools=defaultPool=$LOXILB_PRIVATE_IP/32
#- --monitor
#- --setBGP
- --setLBMode=2
Expand Down
2 changes: 1 addition & 1 deletion cicd/k0s-incluster/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ function create_lb_rule() {
echo "$1: loxicmd create lb ${args[*]}"
$dexec $1 loxicmd create lb ${args[*]}

hook=$($dexec llb1 tc filter show dev eth0 ingress | grep tc_packet_func)
hook=$($dexec $1 tc filter show dev eth0 ingress | grep tc_packet_func)
if [[ $hook != *"tc_packet_func"* ]]; then
echo "ERROR : No hook point found";
exit 1
Expand Down
3 changes: 2 additions & 1 deletion cicd/k0s-incluster/kube-loxilb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ rules:
resources:
- endpoints
- services
- namespaces
- services/status
verbs:
- get
Expand Down Expand Up @@ -111,7 +112,7 @@ spec:
- /bin/kube-loxilb
args:
#- --loxiURL=http://172.17.0.2:11111
- --externalCIDR=192.168.82.100/32
- --cidrPools=defaultPool=192.168.82.100/32
- --setRoles=0.0.0.0
#- --monitor
#- --setBGP
Expand Down
4 changes: 2 additions & 2 deletions cicd/k0s-weave/kube-loxilb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ rules:
resources:
- endpoints
- services
- namespaces
- services/status
verbs:
- get
Expand Down Expand Up @@ -111,8 +112,7 @@ spec:
- /bin/kube-loxilb
args:
- --loxiURL=http://12.12.12.1:11111,http://14.14.14.1:11111
- --externalCIDR=123.123.123.1/24
#- --externalSecondaryCIDRs=124.124.124.1/24,125.125.125.1/24
- --cidrPools=defaultPool=123.123.123.1/24
#- --monitor
#- --setBGP
#- --setLBMode=1
Expand Down
Loading

0 comments on commit 912e864

Please sign in to comment.