Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve CPU cores affinity to NUMA nodes v9 #12359

Closed
wants to merge 10 commits into from
Closed

improve CPU cores affinity to NUMA nodes v9 #12359

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
e9ac062
runmodes: query the active runmode with a function call
Dec 8, 2024
63b0d43
dpdk: move DPDK socket retrieval to utils
Dec 6, 2024
df09fd5
github-ci: install hwloc as a mandatory dependency
Sep 3, 2024
ba957a0
actions: test hwloc build
Dec 19, 2024
0152f53
threading: refactor CPU affinity code
Jan 2, 2025
e29bb2f
util-affinity: move properties of *-cpu-set node one layer up in YAML
Dec 6, 2024
50b652d
threading: transform *-cpu-set nodes from list items to nodes
Dec 6, 2024
b266f3f
threading: support previous threading configuration format
Jan 3, 2025
b9a94be
doc: remove title in threading section with no content
Jan 3, 2025
9d4d5b3
threading: support thread autopinning and interface-specific affinity
Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 64 additions & 1 deletion .github/workflows/builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -258,6 +260,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -353,6 +357,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -504,6 +510,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -601,6 +609,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libtool \
Expand Down Expand Up @@ -693,6 +703,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
hiredis-devel \
jansson-devel \
jq \
Expand Down Expand Up @@ -791,6 +803,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -889,6 +903,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -987,6 +1003,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -1080,6 +1098,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -1165,6 +1185,8 @@ jobs:
gcc-c++ \
git \
hiredis-devel \
hwloc \
hwloc-devel \
jansson-devel \
jq \
libasan \
Expand Down Expand Up @@ -1261,6 +1283,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
libasan \
libtool \
libyaml-devel \
Expand Down Expand Up @@ -1382,6 +1406,8 @@ jobs:
coccinelle \
dpdk-dev \
git \
hwloc \
libhwloc-dev \
jq \
libcap-ng-dev \
libevent-dev \
Expand Down Expand Up @@ -1454,6 +1480,8 @@ jobs:
clang-14 \
curl \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -1591,6 +1619,8 @@ jobs:
llvm-14-dev \
clang-14 \
git \
hwloc \
libhwloc-dev \
jq \
inetutils-ping \
libc++-dev \
Expand Down Expand Up @@ -1696,6 +1726,8 @@ jobs:
llvm-14-dev \
clang-14 \
git \
hwloc \
libhwloc-dev \
jq \
inetutils-ping \
libc++-dev \
Expand Down Expand Up @@ -1835,6 +1867,8 @@ jobs:
cbindgen \
clang-18 \
git \
hwloc \
libhwloc-dev \
jq \
libc++-dev \
libc++abi-dev \
Expand Down Expand Up @@ -1922,6 +1956,8 @@ jobs:
cbindgen \
clang-18 \
git \
hwloc \
libhwloc-dev \
jq \
inetutils-ping \
libc++-dev \
Expand Down Expand Up @@ -2023,6 +2059,8 @@ jobs:
llvm-14-dev \
clang-14 \
git \
hwloc \
libhwloc-dev \
jq \
libc++-dev \
libc++abi-dev \
Expand Down Expand Up @@ -2122,6 +2160,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2205,6 +2245,8 @@ jobs:
apt -y install \
build-essential \
curl \
hwloc \
libhwloc-dev \
libtool \
libpcap-dev \
libnet1-dev \
Expand Down Expand Up @@ -2274,6 +2316,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2358,6 +2402,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
libtool \
libpcap-dev \
libnet1-dev \
Expand Down Expand Up @@ -2423,6 +2469,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2562,6 +2610,8 @@ jobs:
automake \
cargo \
git \
hwloc \
libhwloc-dev \
jq \
libtool \
libpcap-dev \
Expand Down Expand Up @@ -2624,7 +2674,7 @@ jobs:
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-dpdk
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-dpdk --enable-hwloc
- run: make -j ${{ env.CPUS }}
- run: make check
# IDS config
Expand Down Expand Up @@ -2660,6 +2710,8 @@ jobs:
curl \
dpdk-dev \
git \
hwloc \
libhwloc-dev \
jq \
make \
libpcre3 \
Expand Down Expand Up @@ -2763,6 +2815,8 @@ jobs:
cmake \
curl \
git \
hwloc \
libhwloc-dev \
jq \
make \
libpcre3 \
Expand Down Expand Up @@ -2844,6 +2898,8 @@ jobs:
curl \
dpdk-dev \
git \
hwloc \
libhwloc-dev \
jq \
make \
libpcre3 \
Expand Down Expand Up @@ -2928,6 +2984,8 @@ jobs:
ccache \
curl \
git \
hwloc \
libhwloc-dev \
jq \
libpcre2-dev \
libpcap-dev \
Expand Down Expand Up @@ -3001,6 +3059,8 @@ jobs:
ccache \
curl \
git \
hwloc \
libhwloc-dev \
jq \
libpcre2-dev \
libpcap-dev \
Expand Down Expand Up @@ -3065,6 +3125,7 @@ jobs:
cbindgen \
curl \
hiredis \
hwloc \
jansson \
jq \
libmagic \
Expand Down Expand Up @@ -3294,6 +3355,8 @@ jobs:
gcc \
gcc-c++ \
git \
hwloc \
hwloc-devel \
jansson-devel \
libtool \
libyaml-devel \
Expand Down
28 changes: 28 additions & 0 deletions configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,33 @@
exit 1
fi

LIBHWLOC=""
AC_ARG_ENABLE(hwloc,
AS_HELP_STRING([--enable-hwloc], [Enable hwloc support [default=no]]),
[enable_hwloc=$enableval],[enable_hwloc=no])
AS_IF([test "x$enable_hwloc" = "xyes"], [
PKG_CHECK_MODULES([HWLOC], [hwloc >= 2.0.0],
[AC_DEFINE([HAVE_HWLOC], [1], [Define if hwloc library is present and meets version requirements])],
LIBHWLOC="no")

if test "$LIBHWLOC" = "no"; then
echo
echo " ERROR! hwloc library version > 2.0.0 not found, go get it"
echo " from https://www.open-mpi.org/projects/hwloc/ "
echo " or your distribution:"
echo
echo " Ubuntu: apt-get install hwloc libhwloc-dev"
echo " Fedora: dnf install hwloc hwloc-devel"
echo " CentOS/RHEL: yum install hwloc hwloc-devel"
echo
exit 1
else
CFLAGS="${CFLAGS} ${HWLOC_CFLAGS}"
LDFLAGS="${LDFLAGS} ${HWLOC_LIBS}"
enable_hwloc="yes"
fi
])

# libpthread
AC_ARG_WITH(libpthread_includes,
[ --with-libpthread-includes=DIR libpthread include directory],
Expand Down Expand Up @@ -2561,6 +2588,7 @@ SURICATA_BUILD_CONF="Suricata Configuration:
JA4 support: ${enable_ja4}
Non-bundled htp: ${enable_non_bundled_htp}
Hyperscan support: ${enable_hyperscan}
Hwloc support: ${enable_hwloc}
Libnet support: ${enable_libnet}
liblz4 support: ${enable_liblz4}
Landlock support: ${enable_landlock}
Expand Down
12 changes: 6 additions & 6 deletions doc/userguide/capture-hardware/dpdk.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,12 +139,12 @@ management and worker CPU set.
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
- receive-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
- worker-cpu-set:
cpu: [ 2,4,6,8 ]
management-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
receive-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
worker-cpu-set:
cpu: [ 2,4,6,8 ]
...

Interrupt (power-saving) mode
Expand Down
Loading
Loading