From 8bc2d0df2077c9f072d32dc735f7e92878bb8568 Mon Sep 17 00:00:00 2001 From: Adarsh Date: Thu, 29 Aug 2024 00:24:47 +0530 Subject: [PATCH 1/4] Fix broken output text in Code Snippets Signed-off-by: Adarsh --- backends/dpdk/README.md | 2 +- backends/p4tools/modules/testgen/README.md | 4 ++-- docs/CodingStandardPhilosophy.md | 2 +- docs/IR.md | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/backends/dpdk/README.md b/backends/dpdk/README.md index 26a7858f4f2..df47204a944 100644 --- a/backends/dpdk/README.md +++ b/backends/dpdk/README.md @@ -68,7 +68,7 @@ To load the 'spec' file in dpdk follow the instructions in the - Currently, programs written for DPDK target should limit the functionality in Ingress blocks, in case non empty Egress blocks are present it will be ignored by default unless Hidden temporary option `--enableEgress` used. When egress block support gets added to the DPDK target, and compiler can generate separate spec file for non empty ingress and egress blocks then option `--enableEgress` will be removed. - DPDK architecture assumes the following signatures for programmable block of PSA. P4C-DPDK converts the input program to this form. -```P4 +``` P4 parser IngressParser (packet_in buffer, out H parsed_hdr, inout M user_meta); control Ingress (inout H hdr, inout M user_meta); diff --git a/backends/p4tools/modules/testgen/README.md b/backends/p4tools/modules/testgen/README.md index 2c7390abb5c..339427bdb96 100644 --- a/backends/p4tools/modules/testgen/README.md +++ b/backends/p4tools/modules/testgen/README.md @@ -114,7 +114,7 @@ P4Testgen supports the use of custom externs to restrict the breadth of possible #### Restricted Tests `testgen_assume(expr)` will add `expr` as a necessary path constraints to all subsequent execution. For example, for the following snippet -```p4 +``` P4 state parse_ethernet { packet.extract(headers.ethernet); testgen_assume(headers.ethernet.ether_type == 0x800); @@ -130,7 +130,7 @@ only inputs which have `0x800` as Ethertype will be generated. This mode is enab #### Finding Assertion Violations Conversely, `testgen_assert(expr)` can be used to find violations in a particular P4 program. By default, `testgen_assert` behaves like `testgen_assume`. If the flag `--assertion-mode` is enabled, P4Testgen will only generate tests that will cause `expr` to be false and, hence, violate the assertion. For example, for -```p4 +``` P4 state parse_ethernet { packet.extract(headers.ethernet); transition select(headers.ethernet.ether_type) { diff --git a/docs/CodingStandardPhilosophy.md b/docs/CodingStandardPhilosophy.md index 4c2562a0f24..95fe83f9c8d 100644 --- a/docs/CodingStandardPhilosophy.md +++ b/docs/CodingStandardPhilosophy.md @@ -138,7 +138,7 @@ the `boost::format` for the format argument, which has some compatibility for `printf` arguments. These functions handle IR and SourceInfo objects smartly. Here is an example: -```C++ +``` C++ IR::NamedRef *ref; error(ErrorType::ERR_INVALID, "%1%: No header or metadata named '%2%'", ref->srcInfo, ref->name); diff --git a/docs/IR.md b/docs/IR.md index c83dc2b94c7..571e4bd8271 100644 --- a/docs/IR.md +++ b/docs/IR.md @@ -72,7 +72,7 @@ IR tree and accumulates information about the tree but does not modify it, while a Transform pass visits every node, possibly modifying the node or replacing it with some other node -```C++ +``` C++ /* pseudo-code for basic Transform visitor */ visit(node, context=ROOT, visited={}) { if (node in visited) { @@ -152,7 +152,7 @@ As an example for how these are used in the IR visitor routines, the IR::If clas three children to visit -- a predicate and two consequents, with the value of the predicate deciding which consequent to execute. The child visitor for `IR::If` is -```C++ +``` C++ visitor.visit(predicate); clone = visitor.flow_clone(); visitor.visit(ifTrue); From aef7f60775cbf49a9bfc5aa81515b915aea475dd Mon Sep 17 00:00:00 2001 From: Adarsh Date: Fri, 20 Sep 2024 09:42:16 +0530 Subject: [PATCH 2/4] Format spacing & remove empty heading Signed-off-by: Adarsh --- backends/ebpf/README.md | 4 ++-- backends/ebpf/psa/README.md | 2 +- backends/p4tools/modules/testgen/README.md | 2 ++ 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/backends/ebpf/README.md b/backends/ebpf/README.md index 0ee14ed370a..918398c07c1 100644 --- a/backends/ebpf/README.md +++ b/backends/ebpf/README.md @@ -280,7 +280,7 @@ state transition | `goto` statement `extract` | load/shift/mask data from packet buffer #### Translating match-action pipelines -## + P4 Construct | C Translation ----------|------------ table | 2 eBPF tables: second one used just for the default action @@ -385,7 +385,7 @@ The C extern could be also allowed to access packet’s payload, but this featur The custom C extern function should be explicitly declared in the P4 program making use of that extern. For example: -```C +```c /** Declaration of the C extern function. */ extern bool verify_ipv4_checksum(in IPv4_h iphdr); ``` diff --git a/backends/ebpf/psa/README.md b/backends/ebpf/psa/README.md index d45fa482bb1..90ed1c3918f 100644 --- a/backends/ebpf/psa/README.md +++ b/backends/ebpf/psa/README.md @@ -378,7 +378,7 @@ or member. Before action execution, following source code will be generated (and some additional comments to it) for table lookup, which has implementation `ActionSelector`: -```c +``` c struct ingress_as_value * as_value = NULL; // pointer to an action data u32 as_action_ref = value->ingress_as_ref; // value->ingress_as_ref is entry from table (reference) u8 as_group_state = 0; // which map contains action data diff --git a/backends/p4tools/modules/testgen/README.md b/backends/p4tools/modules/testgen/README.md index 339427bdb96..d1709749d16 100644 --- a/backends/p4tools/modules/testgen/README.md +++ b/backends/p4tools/modules/testgen/README.md @@ -115,6 +115,7 @@ P4Testgen supports the use of custom externs to restrict the breadth of possible #### Restricted Tests `testgen_assume(expr)` will add `expr` as a necessary path constraints to all subsequent execution. For example, for the following snippet ``` P4 + state parse_ethernet { packet.extract(headers.ethernet); testgen_assume(headers.ethernet.ether_type == 0x800); @@ -131,6 +132,7 @@ only inputs which have `0x800` as Ethertype will be generated. This mode is enab Conversely, `testgen_assert(expr)` can be used to find violations in a particular P4 program. By default, `testgen_assert` behaves like `testgen_assume`. If the flag `--assertion-mode` is enabled, P4Testgen will only generate tests that will cause `expr` to be false and, hence, violate the assertion. For example, for ``` P4 + state parse_ethernet { packet.extract(headers.ethernet); transition select(headers.ethernet.ether_type) { From 6074b6f1a245cf30bc20594198164f1672137940 Mon Sep 17 00:00:00 2001 From: Adarsh Date: Fri, 25 Oct 2024 23:14:38 +0530 Subject: [PATCH 3/4] docs: temp fix for code snippet rendering in eBPF Markdown Signed-off-by: Adarsh --- backends/ebpf/psa/README.md | 281 ++++++++++++++++++++++++++++++++++-- 1 file changed, 267 insertions(+), 14 deletions(-) diff --git a/backends/ebpf/psa/README.md b/backends/ebpf/psa/README.md index 90ed1c3918f..0b5a8e32769 100644 --- a/backends/ebpf/psa/README.md +++ b/backends/ebpf/psa/README.md @@ -106,6 +106,22 @@ The `MAX_RESUBMIT_DEPTH` variable specifies maximum number of resubmit operation The `resubmit` flag defines whether the `tc-ingress` program should enter next iteration (resubmit) or break the loop. The pseudocode looks as follows: + + ```c int i = 0; int ret = TC_ACT_UNSPEC; @@ -117,6 +133,9 @@ for (i = 0; i < MAX_RESUBMIT_DEPTH; i++) { } } ``` + The same mechanism for RESUBMIT is used in the TC-based and XDP-based design. @@ -286,6 +305,63 @@ the best match is overwritten. The algorithm exists when there is no more tuples The snippet below shows the C code generated by the PSA-eBPF compiler for a lookup into a ternary table. The steps are explained below. + + + ```c struct ingress_tbl_ternary_1_key key = {}; key.field0 = hdr->ipv4.dstAddr; @@ -337,6 +413,9 @@ if (val && val->has_next != 0) { // (6): go to default action if value == NULL ``` + The description of annotated lines: 1. The algorithm starts to iterate over the ternary masks map. The loop is bounded by the `MAX_INGRESS_TBL_TERNARY_1_KEY_MASKS` which is configured by `--max-ternary-masks` compiler option (defaults to 128). @@ -378,6 +457,61 @@ or member. Before action execution, following source code will be generated (and some additional comments to it) for table lookup, which has implementation `ActionSelector`: + + ``` c struct ingress_as_value * as_value = NULL; // pointer to an action data u32 as_action_ref = value->ingress_as_ref; // value->ingress_as_ref is entry from table (reference) @@ -428,6 +562,9 @@ if (as_group_state == 0) { as_value = BPF_MAP_LOOKUP_ELEM(ingress_as_defaultActionGroup, &ebpf_zero); // (7) } ``` + Description of marked lines: 1. Detect if a reference is a group reference. When the `_is_group_ref` field is non-zero, the reference is assumed to be a group reference. @@ -496,33 +633,74 @@ The PSA implemented for eBPF backend is verified to work with the kernel version Moreover, make sure that the BPF filesystem is mounted under `/sys/fs/bpf`. Also, make sure you have the following packages installed: - + + ```bash $ sudo apt install -y clang llvm libelf-dev ``` - + You should also install a static `libbpf` library. Run the following commands: - + + ```bash $ python3 backends/ebpf/build_libbpf ``` - + ### Compilation You can compile a P4-16 PSA program for eBPF in a single step using: + + ```bash make -f backends/ebpf/runtime/kernel.mk BPFOBJ=out.o P4FILE=.p4 P4C=p4c-ebpf psa ``` + You can also perform compilation step by step: + + ``` $ p4c-ebpf --arch psa --target kernel -o out.c .p4 $ clang -Ibackends/ebpf/runtime -Ibackends/ebpf/runtime/usr/include -O2 -g -c -emit-llvm -o out.bc out.c $ llc -march=bpf -mcpu=generic -filetype=obj -o out.o out.bc ``` - + Note that you can use `-mcpu` flag to define the eBPF instruction set. Visit [this blog post](https://pchaigno.github.io/bpf/2021/10/20/ebpf-instruction-sets.html) to learn more about eBPF instruction sets. The above steps generate `out.o` BPF object file that can be loaded to the kernel. @@ -531,9 +709,20 @@ The above steps generate `out.o` BPF object file that can be loaded to the kerne Supposing we want to use a packet recirculation we have to specify the `PSA_PORT_RECIRCULATE` port. We can use `-DPSA_PORT_RECIRCULATE=` Clang flag via `kernel.mk` + + ```bash make -f backends/ebpf/runtime/kernel.mk BPFOBJ=out.o ARGS="-DPSA_PORT_RECIRCULATE=" P4FILE=.p4 P4C=p4c-ebpf psa ``` + or directly: `clang ... -DPSA_PORT_RECIRCULATE= ...`, where `RECIRCULATE_PORT_IDX` is a number of a `psa_recirc` interface (this number can be obtained from `ip -n switch link`). @@ -549,17 +738,36 @@ To install the CLI tool, follow the guide in [the NIKSS repository](https://gith (e.g., default actions) will only work when using `nikss-ctl`. To load eBPF programs generated by P4-eBPF compiler run: - + + ```bash nikss-ctl pipeline load id out.o ``` - + `PIPELINE-ID` is a user-defined value used to uniquely identify PSA-eBPF pipeline (we are going to support for multiple PSA-eBPF pipelines running in parallel). In the next step, for each interface that should be attached to PSA-eBPF run: - + + ```bash nikss-ctl add-port pipe dev ``` + ## Running PTF tests @@ -567,23 +775,51 @@ PSA implementation for eBPF backend is covered by a set of PTF tests that verify The test scripts, PTF test cases and test P4 programs are located under `backends/ebpf/tests`. The tests must be executed from this directory. To run all PTF tests: - + + ``` sudo ./test.sh ``` + You can also specify a single PTF test to run: - + + ``` sudo ./test.sh test.BridgedMetadataPSATest ``` - + It might be also useful to enable tracing for troubleshooting with `bpftool prog tracelog`: - + + ``` sudo ./test.sh --trace=on ``` - + ## Troubleshooting The PSA implementation for eBPF backend generates standard BPF objects that can be inspected using `bpftool`. @@ -591,7 +827,21 @@ The PSA implementation for eBPF backend generates standard BPF objects that can To troubleshoot PSA-eBPF program you will probably need `bpftool`. Follow the steps below to install it. You should be able to see `bpftool help`: - + + ```bash $ bpftool help Usage: bpftool [OPTIONS] OBJECT { COMMAND | help } @@ -602,6 +852,9 @@ $ bpftool help OPTIONS := { {-j|--json} [{-p|--pretty}] | {-f|--bpffs} | {-m|--mapcompat} | {-n|--nomount} } ``` + Refer to [the bpftool guide](https://manpages.ubuntu.com/manpages/focal/man8/bpftool-prog.8.html) for more examples how to use it. From 333770d25a1e83f77ca22b07db973ee7c7aa163b Mon Sep 17 00:00:00 2001 From: Adarsh Date: Tue, 29 Oct 2024 17:05:04 +0530 Subject: [PATCH 4/4] docs: update C/C++ code block syntax in README for Doxygen This syntax is processed correctly by Doxygen Signed-off-by: Adarsh --- backends/ebpf/README.md | 2 +- backends/ebpf/psa/README.md | 2 +- backends/tofino/bf-p4c/phv/slicing/README.md | 4 ++-- backends/tofino/bf-p4c/phv/solver/README.md | 2 +- docs/CodingStandardPhilosophy.md | 2 +- docs/IR.md | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/backends/ebpf/README.md b/backends/ebpf/README.md index 918398c07c1..38db453fe77 100644 --- a/backends/ebpf/README.md +++ b/backends/ebpf/README.md @@ -425,7 +425,7 @@ clang -O2 -include C-EXTERN-FILE.c -target bpf -c OUTPUT.c -o OUTPUT.o * BPF maps can be defined inside the C extern function to provide statefulness. BPF map can be defined as follows: - ```C + ```c REGISTER_START() REGISTER_TABLE(, BPF_MAP_TYPE_HASH, , >, ) REGISTER_END() diff --git a/backends/ebpf/psa/README.md b/backends/ebpf/psa/README.md index 0b5a8e32769..191c7600e90 100644 --- a/backends/ebpf/psa/README.md +++ b/backends/ebpf/psa/README.md @@ -512,7 +512,7 @@ if (as_group_state == 0) { -``` c +```c struct ingress_as_value * as_value = NULL; // pointer to an action data u32 as_action_ref = value->ingress_as_ref; // value->ingress_as_ref is entry from table (reference) u8 as_group_state = 0; // which map contains action data diff --git a/backends/tofino/bf-p4c/phv/slicing/README.md b/backends/tofino/bf-p4c/phv/slicing/README.md index 17e63129bb4..8639cb5ae13 100644 --- a/backends/tofino/bf-p4c/phv/slicing/README.md +++ b/backends/tofino/bf-p4c/phv/slicing/README.md @@ -72,7 +72,7 @@ split. The algorithm is as follows: 3. Restore the state by removing *head* from result and *tail* from header. Pseudo codes of the DFS: -```c++ +```cpp // global states Headers = [A, B, C, D]; Result = [] @@ -397,7 +397,7 @@ enhancing our after-split constraint. Previously, an after-split constraint can the 'exact' requirement. We introduce a less restricted form of after-split constraint, which specifies a minimum size of the list that a metadata resides in. -```c++ +```cpp struct AfterSplitConstraint { enum class ConstraintType { EXACT = 0, // must be placed in container of the size. diff --git a/backends/tofino/bf-p4c/phv/solver/README.md b/backends/tofino/bf-p4c/phv/solver/README.md index 2ad5b376437..53fac0d52a9 100644 --- a/backends/tofino/bf-p4c/phv/solver/README.md +++ b/backends/tofino/bf-p4c/phv/solver/README.md @@ -9,7 +9,7 @@ check and generate instruction based on the PHV allocation. ## Example of using Z3 an example of using z3 to validate deposit field. Unfortunately it's about 20ms per invocation, which is too slow for us. -``` c++ +```cpp boost::optional ActionMoveSolver::run_deposit_field_z3_solver( const ContainerID dest, const std::vector& src1, const std::vector& src2) const { diff --git a/docs/CodingStandardPhilosophy.md b/docs/CodingStandardPhilosophy.md index 95fe83f9c8d..8f2bb25bda5 100644 --- a/docs/CodingStandardPhilosophy.md +++ b/docs/CodingStandardPhilosophy.md @@ -138,7 +138,7 @@ the `boost::format` for the format argument, which has some compatibility for `printf` arguments. These functions handle IR and SourceInfo objects smartly. Here is an example: -``` C++ +```cpp IR::NamedRef *ref; error(ErrorType::ERR_INVALID, "%1%: No header or metadata named '%2%'", ref->srcInfo, ref->name); diff --git a/docs/IR.md b/docs/IR.md index 571e4bd8271..4ea87e3eaca 100644 --- a/docs/IR.md +++ b/docs/IR.md @@ -72,7 +72,7 @@ IR tree and accumulates information about the tree but does not modify it, while a Transform pass visits every node, possibly modifying the node or replacing it with some other node -``` C++ +```cpp /* pseudo-code for basic Transform visitor */ visit(node, context=ROOT, visited={}) { if (node in visited) { @@ -152,7 +152,7 @@ As an example for how these are used in the IR visitor routines, the IR::If clas three children to visit -- a predicate and two consequents, with the value of the predicate deciding which consequent to execute. The child visitor for `IR::If` is -``` C++ +```cpp visitor.visit(predicate); clone = visitor.flow_clone(); visitor.visit(ifTrue);