Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get multiport-eswitch mode from metalnet rundir #270

Merged
merged 1 commit into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Build the manager binary
FROM --platform=$BUILDPLATFORM golang:1.23 as builder
FROM --platform=$BUILDPLATFORM golang:1.23 AS builder

ARG GOARCH=''

Expand Down
22 changes: 22 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,25 @@

## CRD usage
* [Usage](./usage/crd_usage.md)

## Multiport-eswitch mode
When running dpservice with Mellanox in multiport-eswitch mode, it is important to tell metalnet about it:
```
metalnet --multiport-eswitch
```
or (overrides the above)
```
echo -n "eswitch" > /var/lib/metalnet/mode
```
This changes the way metalnet generates identifiers for virtual function representors that are sent over to dpservice.

If pf1-proxy is also in use, it is important to mark it as used in the metalnet VF database:
```
mkdir -p /var/lib/metalnet/netfns/claims
echo -n "$pf1_proxy_vf_pci" > /var/lib/metalnet/netfns/claims/00000001-0000-4000-0000-000000000000
```
Where the `$pf1_proxy_vf_pci` is the PCI address of the VF representor for pf1-proxy. This should be the only VF using `mlx5_core` driver instead of the `vfio-pci` driver. One of many ways to retrieve such address is as follows:
```
pf1_proxy_vf_name=$(/opt/local/bin/dpdk-devbind.py -s | grep "mlx5Gen Virtual Function" | grep "drv=mlx5_core" | awk -F'if=' '{print $2}' | awk '{print $1}')
pf1_proxy_vf_pci=$(/opt/local/bin/dpdk-devbind.py -s | grep "mlx5Gen Virtual Function" | grep $pf1_proxy_vf_name | awk '{print $1}')
```
11 changes: 10 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ func main() {
flag.BoolVar(&enableIPv6Support, "enable-ipv6", false, "Enable IPv6 support")
flag.IntVar(&publicVNI, "public-vni", 100, "Virtual network identifier used for public routing announcements.")
flag.IPVar(&routerAddress, "router-address", net.IP{}, "The address of the next router.")
flag.BoolVar(&multiportEswitchMode, "multiport-eswitch", false, "Enable multiport eswitch support")
flag.BoolVar(&multiportEswitchMode, "multiport-eswitch", false, "Enable multiport eswitch support (can be overridden in metalnet-dir)")
flag.BoolVar(&enableLeaderElection, "leader-elect", false,
"Enable leader election for controller manager. "+
"Enabling this will ensure there is only one active controller manager.")
Expand All @@ -125,6 +125,15 @@ func main() {
log.SetLevel(log.DebugLevel)
}

// detect multiport-eswitch mode automatically (overrides command-line)
// Usage example: 'echo "eswitch" > /var/lib/metalnet/mode'
modeFilePath := filepath.Join(metalnetDir, "mode")
content, err := os.ReadFile(modeFilePath)
if err == nil {
multiportEswitchMode = strings.TrimSpace(string(content)) == "eswitch"
}
setupLog.Info(fmt.Sprintf("Multiport Eswitch mode set to: %v", multiportEswitchMode))

defaultRouterAddr.PublicVNI = uint32(publicVNI)
defaultRouterAddr.SetBySubsciption = false

Expand Down
Loading