Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Use IPDK-container to speed-up the DPDK-PTF CI #4212

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[WIP] Use IPDK-container to speed-up the DPDK-PTF CI #4212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a043238
Use IPDK-container to Spped-up DPDK-PTF CI
Hoooao Oct 24, 2023
84a9b1e
Use Ccache in Container
Hoooao Oct 27, 2023
0a21c9c
CI to main
Hoooao Nov 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 18 additions & 74 deletions .github/workflows/ci-dpdk-ptf-p4testgen-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: "p4c-dpdk-ptf-p4testgen-tests"
name: "p4c-dpdk-ptf-container-p4testgen-tests"

on:
push:
Expand All @@ -13,43 +13,17 @@ concurrency:
group: p4c_dpdk_ptf_ci-${{ github.ref }}
cancel-in-progress: true

# Envs for infrap4d and dpdk libs
# IPDK_INSTALL_DIR is put to a default searching directory for
# visibility of different libs like libprotobuf
env:
IPDK_INSTALL_DIR: ${{github.workspace}}/ipdk_install
CMAKE_UNITY_BUILD: ON
ENABLE_TEST_TOOLS: ON
INSTALL_DPDK: ON
INSTALL_BMV2: OFF
INSTALL_EBPF: OFF
IMAGE_TYPE: test
CTEST_PARALLEL_LEVEL: 4

jobs:
build_p4dpdk_ubuntu:
runs-on: ubuntu-22.04
steps:
# TODO: Cache for container
- name: ccache
uses: hendrikmuhs/ccache-action@v1
with:
key: ptf-${{ runner.os }}-test
max-size: 1000M

- name: Checkout DPDK-target
uses: actions/checkout@v3
with:
repository: p4lang/p4-dpdk-target
path: p4sde
submodules: recursive

- name: Checkout ipdk-recipe
uses: actions/checkout@v3
with:
repository: ipdk-io/networking-recipe
path: ipdk.recipe
submodules: recursive

- name: Checkout P4C
uses: actions/checkout@v3
with:
Expand All @@ -58,56 +32,26 @@ jobs:

- name: Install hugepages and set transparent gugepages to madvise
run: |
sudo apt-get install libhugetlbfs-bin
sudo apt-get install libhugetlbfs-bin -y
sudo hugeadm --thp-madvise

- name: Install DPDK dependencies
working-directory: p4sde/tools/setup
run: |
sudo apt update -y
python3 install_dep.py

- name: Compile p4sde dpdk target
working-directory: p4sde
run: |
mkdir ${GITHUB_WORKSPACE}/install
./autogen.sh
./configure --prefix=$IPDK_INSTALL_DIR
make
make install

- name: Build infrap4d dependencies
working-directory: ipdk.recipe
run: |
echo "Install infrap4d dependencies"
sudo apt install libatomic1 libnl-route-3-dev openssl
pip3 install -r requirements.txt
cd setup
echo "Build infrap4d dependencies"
cmake -B build -DCMAKE_INSTALL_PREFIX="$IPDK_INSTALL_DIR"
cmake --build build

- name: Build infrap4d
working-directory: ipdk.recipe
run: |
./make-all.sh --target=dpdk --no-krnlmon --no-ovs -S $IPDK_INSTALL_DIR -D $IPDK_INSTALL_DIR --prefix=$IPDK_INSTALL_DIR
- name: Checkout DPDK-target
uses: actions/checkout@v3
with:
repository: p4lang/p4-dpdk-target
path: p4sde
submodules: recursive

- name: Build P4C with only the DPDK backend
working-directory: p4c
- name: Build IPDK
run: |
export CMAKE_FLAGS+="-DENABLE_BMV2=OFF "
export CMAKE_FLAGS+="-DENABLE_EBPF=OFF "
export CMAKE_FLAGS+="-DENABLE_UBPF=OFF "
export CMAKE_FLAGS+="-DENABLE_GTESTS=OFF "
export CMAKE_FLAGS+="-DENABLE_P4TEST=OFF "
export CMAKE_FLAGS+="-DENABLE_P4TC=OFF "
export CMAKE_FLAGS+="-DENABLE_P4C_GRAPHS=OFF "
export CMAKE_FLAGS+="-DENABLE_TEST_TOOLS=ON "
export CMAKE_FLAGS+="-DIPDK_INSTALL_DIR=$IPDK_INSTALL_DIR "
tools/ci-build.sh
cp p4c/tools/dpdk_ptf.Dockerfile .
docker build -t dpdk_ptf -f dpdk_ptf.Dockerfile .

# hugepage has to be set in the same shell session
- name: Run DPDK PTF tests using P4Testgen
working-directory: p4c/build
run: |
sudo $IPDK_INSTALL_DIR/sbin/set_hugepages.sh
sudo -E ctest -j1 --output-on-failure --schedule-random -R testgen-p4c-pna-ptf
sudo -E docker run --privileged -w /root/p4c/build dpdk_ptf bash -c "../../scripts/set_hugepages.sh && ctest -j1 --output-on-failure -R testgen-p4c-pna-ptf"
rm -rf ./.ccache
docker cp $(docker create --rm dpdk_ptf):/root/.ccache .


49 changes: 49 additions & 0 deletions tools/dpdk_ptf.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
FROM ghcr.io/ipdk-io/ipdk-ubuntu2004-x86_64:ipdk_v23.07
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tools might be a strange folder for this file. Maybe create a docker folder?


# Prep
ENV IPDK_INSTALL_DIR=/root/ipdk_install
ENV PATH="${PATH}:/root/.local/bin"
WORKDIR /root
RUN rm -rf p4*
COPY ./p4c /root/p4c
COPY ./p4sde /root/p4sde
COPY ./.ccache /root/.ccache
RUN apt-get update && apt-get install rsync git ccache libhugetlbfs-bin -y

# Update GCC to 11 -> GCC-9 cause config problem, remove after base OS is updated to 22.04
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to wait for a Ubuntu 22.04 docker version. @jafingerhut
anyone we can poke there?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am guessing you are asking whether someone at Intel might publish a docker image with IPDK software already compiled and installed, rather than following instructions like these? https://github.com/ipdk-io/ipdk/blob/main/build/networking/README_DOCKER.md

If so, let me know and I can try pinging a couple of people to find out if they have any intention of supporting such a thing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah apologize, let me clarify:
https://github.com/orgs/ipdk-io/packages?repo_name=ipdk
We have borrowed a docker image from this list of images. However, only Ubuntu 20.04 is available. The DPDK target is only tested with Ubuntu 22.04. It would be nice to have a 22.04 version to unify things.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my docker-fu is still weak, so asking to clarify. You are running these tests on a base OS that is Ubuntu 22.04, and a docker image created using Ubuntu 20.04 does not work there, and you would prefer a docker image that was built specifically for a base OS of Ubuntu 22.04?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker image is built using a base OS that is Ubuntu 22.04, but the docker image itself is 20.04. That is normally fine, but the p4-dpdk-target itself is built with Ubuntu 22.04. To get it to work on Ubuntu 20.04 requires a bunch of workarounds. It would be nice to remove this confounding factor.

We thought we fixed the nondeterministic failures but that is sadly still not the case.

RUN sudo apt install build-essential manpages-dev software-properties-common -y
RUN sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
RUN sudo apt update && sudo apt install gcc-11 g++-11 -y
RUN sudo update-alternatives --remove-all cpp
RUN sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 90 --slave /usr/bin/g++ g++ /usr/bin/g++-9 --slave /usr/bin/gcov gcov /usr/bin/gcov-9 --slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-9 --slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-9 --slave /usr/bin/cpp cpp /usr/bin/cpp-9 \
&& sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 110 --slave /usr/bin/g++ g++ /usr/bin/g++-11 --slave /usr/bin/gcov gcov /usr/bin/gcov-11 --slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-11 --slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-11 --slave /usr/bin/cpp cpp /usr/bin/cpp-11;

# p4sde (tdi requires gcc-11)
WORKDIR /root/p4sde
RUN pip3 install distro && python3 ./tools/setup/install_dep.py
RUN ./autogen.sh && ./configure --prefix=$IPDK_INSTALL_DIR \
&& make -j4 && make install

# infrap4d deps
WORKDIR /root
RUN sudo apt install libatomic1 libnl-route-3-dev openssl -y
RUN rsync -avh ./networking-recipe/deps_install/* $IPDK_INSTALL_DIR \
&& rsync -avh ./networking-recipe/install/* $IPDK_INSTALL_DIR

# P4C
ENV CMAKE_UNITY_BUILD="ON"
ENV ENABLE_TEST_TOOLS="ON"
ENV INSTALL_DPDK="ON"
ENV INSTALL_BMV2="ON"
ENV INSTALL_EBPF="OFF"
ENV IMAGE_TYPE="test"
ENV CTEST_PARALLEL_LEVEL="4"
ENV CMAKE_FLAGS="-DENABLE_P4TC=OFF -DENABLE_BMV2=OFF -DENABLE_EBPF=OFF -DENABLE_UBPF=OFF -DENABLE_GTESTS=OFF -DENABLE_P4TEST=OFF -DENABLE_P4C_GRAPHS=OFF -DIPDK_INSTALL_DIR=$IPDK_INSTALL_DIR "
WORKDIR /root/p4c
RUN apt-get install python3-dev -y # Required by packges in ci-build, remove after base OS is updated to 22.04
Hoooao marked this conversation as resolved.
Show resolved Hide resolved
RUN tools/ci-build.sh

# Disable github aciton default THP setting
# CMD ["sudo", "hugeadm", "--thp-madvise "]


Loading