This software is an experimental partial implementation of the IEEE 802.1CB Frame Replication and Elimination for Reliability standard. The implementation uses the XDP packet processing subsystem of the Linux kernel, which can be configured with BPF.
The details of the experiment are discussed in the following research paper:
Lightweight Implementation of Per-packet Service Protection in eBPF/XDP (pdf | arxiv | slides)
Cite as:
@misc{fejes2023lightweight,
title={Lightweight Implementation of Per-packet Service Protection in eBPF/XDP},
author={Ferenc Fejes and Ferenc Orosi and Balázs Varga and János Farkas},
booktitle={Netdev 0x17, THE Technical Conference on Linux Networking},
year={2023},
eprint={2312.07152},
archivePrefix={arXiv},
primaryClass={cs.NI},
url={https://netdevconf.info/0x17/25}
}
Debian based GNU/Linux distribution is preferred. Tested with Debian Bookworm and Ubuntu 23.04, 23.10.
sudo apt install build-essential gcc-multilib clang llvm linux-tools-common bpftool libbpf-dev
Note: libbpf version must be at least 1.3.0 on Ubuntu 23.04.
cd src
make
Usage: xdpfrer [OPTION...]
-e, --egress=WORD Egress interface in IFNAME:VID format (Required)
-i, --ingress=WORD Ingress interface in IFNAME:VID format (Required)
-m, --mode=WORD Mode: repl or elim (Required)
-n, --not Not adding or removing R-tag. (Optional)
-q, --quiet Quiet output. (Optional)
-?, --help Give this help list
--usage Give a short usage message
Important: if multiple --egress
used, mode must be replication (--mode=repl
) and only one --ingress
interface can be set.
Similarly, if the mode is elimination (-m elim
), multiple --ingress
but only one --egress
parameter are allowed.
For example in the xdpfrer -m repl -i beth0:20 -e enp4s0:66 -e enp7s0:67
command can be interpreted as:
Packets with VLAN ID 20
on the beth0
interface are replicated to enp4s0
and enp7s0
interfaces with VLAN ID 66
and 67
respectively.
.
├── README.md
├── src
│ ├── common.h // basic data structures and defines
│ ├── Makefile // GNU make file
│ ├── xdpfrer.bpf.c // XDP programs and BPF map definitions
│ └── xdpfrer.c // configure and load the BPF part to the kernel
└── test
├── measurement.py // All-in-one testing and plotting script
├── physical.env // Network config/environment for real testbed
└── veth.env // Full network config for veth/namespace based testbed
physical.env
and veth.env
contains this environment. Obviously you can modify vlans when you start running xdpfrer instances.
┌───────────────────────────────────────────────────┐
│ nsx │
│ │
│┌──────────┬────────┐ ┌────────┬──────────┐│
┌─────────────┐ ││ │ enp3s0 ├────55───▶ enp4s0 │ ││ ┌─────────────┐
│ tx │ ││ │ ◀────66───┐ │ ││ │ lx │
│ ┌────────┤ │├────────┐ └────────┤ ├────────┘ ┌────────┤│ ├────────┐ │
│ │ teth0 ◀━━10━┿▶ aeth0 │ │ │ │ beth0 ◀┿━20━━▶ leth0 │ │
│ │10.0.0.1│ │├────────┘ ┌────────┤ ├────────┐ └────────┤│ │10.0.0.2│ │
│ └────────┤ ││ │ enp6s0 ├────56───▶ enp7s0 │ ││ ├────────┘ │
└─────────────┘ ││ │ ◀────67───┐ │ ││ └─────────────┘
│└──────────┴────────┘ └────────┴──────────┘│
│ │
└───────────────────────────────────────────────────┘
For basic usage a GNU/Linux system required with BPF and XDP support.
- Open some terminal and source the
env
file which configure the whole network: network namespaces acting as talker, switch and listener, including the virtual interfaces and links. - Run the
xdpfrer
instances inside the switch namespace (use thensx
alias) - Open up another root terminal, source the
env
file, and start a ping command from talker to listener - If everything OK, the ping successful and the XDP forwarding works properly
- To cleanup, press
Ctrl+D
or typeexit
to exit from terminals. The last terminal cleanup the environment
The commands:
# 1.
cd test
sudo su
source veth.env
# 2.1 (tx -> lx)
nsx ../src/xdpfrer -m repl -i aeth0:10 -e enp3s0:55 -e enp6s0:56
nsx ../src/xdpfrer -m elim -i enp4s0:55 -i enp7s0:56 -e beth0:20
# 2.2 (lx -> tx)
nsx ../src/xdpfrer -m repl -i beth0:20 -e enp4s0:66 -e enp7s0:67
nsx ../src/xdpfrer -m elim -i enp3s0:66 -i enp6s0:67 -e aeth0:10
# 3.
cd test
sudo su
source veth.env
tx ping 10.0.0.2 -c 4
# PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
# 64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.044 ms
# 64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.056 ms
# 64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.055 ms
# 64 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.057 ms
# 4.
# (veth.env) root:test# nsx ../src/xdpfrer -m repl -i aeth0:10 -e enp3s0:55 -e enp6s0:56
# Config replication on interface aeth0 (ifindex: 2) match vlan 10
# Received packets: 0
# Received packets: 1
# Received packets: 2
# Received packets: 3
# Received packets: 4
# Received packets: 4
# Received packets: 4
# (veth.env) root:test# nsx ../src/xdpfrer -m elim -i enp4s0:55 -i enp7s0:56 -e beth0:20
# Config recovery on iface enp4s0 (ifindex: 3) match vlan 20
# Config recovery on iface enp7s0 (ifindex: 5) match vlan 20
# Passed 0, Dropped 0
# Passed 1, Dropped 1
# Passed 2, Dropped 2
# Passed 3, Dropped 3
# Passed 4, Dropped 4
# Passed 4, Dropped 4
# 5.
Ctrl+C # in xdpfrer terminal
Ctrl+D # in both terminal
For advanced usage, take a look at the test/measurement.py script. It is possible to run XDP FRER on a real, physical testbed, just change the interface names and VLAN IDs and the script accordingly.
First, run common tests or error tests:
python3 measurement.py test
or
python3 measurement.py error
After that, generate txt
files from pcap
files:
python3 measurement.py data
Last but not least, generate plots:
python3 measurement.py plot
With questions regarding usage, bugs and evaluation, please contact Ferenc Fejes <[email protected]>.