Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to make a OOT mlx5_core.ko livepatch because of "Compiler lacks asm-goto support.." #1429

Open
Heathcliff4689 opened this issue Dec 17, 2024 · 2 comments

Comments

@Heathcliff4689
Copy link

Heathcliff4689 commented Dec 17, 2024

Environment
OS: Centos 7.3
Kernel: 4.18.0
Kpatch: 0.9.9
GCC:
"
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --enable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --disable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
"

We have upgrade a Centos 7.3's gcc from 4.8.5 to 8.3.1,and have succeeded to compile kernel 4.18 & mlnx-ofa_kernel-5.2-OFED.5.2.1.0.4.1 in the past.

Problem
While we make the kpatch-cc scripts as a prefix of gcc (because of kpatch-build command),the compiling of mlnx-ofa_kernel-5.2-OFED.5.2.1.0.4.1 will fail with "Compiler lacks asm-goto support.."

Step to reproduce
./kpatch-build -s /root/rpmbuild/BUILD/kernel-4.18.0-193.3.0.0/linux-4.18.0-193.3.0.0.el7.x86_64 --oot-module ~/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko --oot-module-src ~/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default/ -t all tmp.patch

cat /root/.kpatch/build.log
"
checking file drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c
patching file drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c
cd ofed_scripts/utils && python setup.py build && cd -
running build
running build_py
running build_scripts
/root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default
Building kernel modules
Kernel version: 4.18.0-193.3.0.0.el7.x86_64
Modules directory: /lib/modules/4.18.0-193.3.0.0.el7.x86_64/extra/mlnx-ofa_kernel
Kernel build: /lib/modules/4.18.0-193.3.0.0.el7.x86_64/build
Kernel sources: /lib/modules/4.18.0-193.3.0.0.el7.x86_64/build
env CWD=/root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default BACKPORT_INCLUDES=
make -C /lib/modules/4.18.0-193.3.0.0.el7.x86_64/build M="/root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default"
V=0 KBUILD_NOCMDDEP=1 CC=/root/kpatch-0.9.9/kpatch-build/kpatch-cc gcc
CONFIG_COMPAT_VERSION=
CONFIG_COMPAT_KOBJECT_BACKPORT=
CONFIG_MEMTRACK=
CONFIG_DEBUG_INFO=y
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_CORE_DUMMY=
CONFIG_INFINIBAND_IPOIB=m
CONFIG_IPOIB_VERSION=
CONFIG_INFINIBAND_IPOIB_CM=y
CONFIG_IPOIB_ALL_MULTI=
CONFIG_INFINIBAND_MLNX_BX=
CONFIG_INFINIBAND_SRP=
CONFIG_INFINIBAND_SRP_DUMMY=m
CONFIG_RDMA_RXE=
CONFIG_RDMA_RXE_DUMMY=m
CONFIG_NVME_CORE=
CONFIG_NVME_HOST_WITHOUT_FC=
CONFIG_BLK_DEV_NVME=
CONFIG_NVME_FABRICS=
CONFIG_NVME_FC=
CONFIG_NVME_RDMA=
CONFIG_NVME_TCP=
CONFIG_NVME_MULTIPATH=
CONFIG_NVME_HOST_DUMMY=m
CONFIG_NVME_TARGET=
CONFIG_NVME_TARGET_LOOP=
CONFIG_NVME_TARGET_RDMA=
CONFIG_NVME_TARGET_TCP=
CONFIG_NVME_TARGET_FC=
CONFIG_NVME_TARGET_FCLOOP=
CONFIG_NVME_TARGET_DUMMY=m
CONFIG_SCSI_SRP_ATTRS=
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_INFINIBAND_USER_ACCESS=m
CONFIG_INFINIBAND_USER_ACCESS_UCM=m
CONFIG_INFINIBAND_USER_MEM=y
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_ADDR_TRANS_CONFIGFS=y
CONFIG_INFINIBAND_IPOIB_DEBUG=y
CONFIG_INFINIBAND_ISERT=
CONFIG_INFINIBAND_ISERT_DUMMY=m
CONFIG_INFINIBAND_ISER=
CONFIG_INFINIBAND_ISER_DUMMY=m
CONFIG_SCSI_ISCSI_ATTRS=
CONFIG_ISCSI_TCP=
CONFIG_INFINIBAND_IPOIB_DEBUG_DATA=
CONFIG_INFINIBAND_SDP_SEND_ZCOPY=
CONFIG_INFINIBAND_SDP_RECV_ZCOPY=
CONFIG_INFINIBAND_SDP_DEBUG=
CONFIG_INFINIBAND_SDP_DEBUG_DATA=
CONFIG_INFINIBAND_MADEYE=
CONFIG_MLXFW=m
CONFIG_MLX5_CORE=m
CONFIG_MLX5_ACCEL=y
CONFIG_MLX5_EN_TLS=y
CONFIG_MLX5_TLS=y
CONFIG_MLX5_FPGA_TLS=y
CONFIG_MLX5_FPGA_IPSEC=y
CONFIG_MLX5_IPSEC=y
CONFIG_MLX5_EN_IPSEC=y
CONFIG_MLX5_FPGA=y
CONFIG_MLX5_FPGA_TOOLS=m
CONFIG_MLX5_CORE_EN=y
CONFIG_MLX5_CORE_EN_DCB=y
CONFIG_MLX5_EN_ARFS=y
CONFIG_MLX5_EN_RXNFC=y
CONFIG_MLX5_MPFS=y
CONFIG_MLX5_ESWITCH=y
CONFIG_MLX5_CLS_ACT=y
CONFIG_MLX5_TC_CT=n
CONFIG_MLX5_SW_STEERING=y
CONFIG_MLX5_CORE_IPOIB=y
CONFIG_MLX5_MDEV=
CONFIG_MLNX_VFIO_MDEV=
CONFIG_MLNX_VFIO_MDEV_DEVICE=
CONFIG_MLX5_INFINIBAND=m
CONFIG_MLX5_DEBUG=y
CONFIG_SUNRPC_XPRT_RDMA=
CONFIG_SUNRPC_XPRT_RDMA_DUMMY=m
CONFIG_DTRACE=
CONFIG_CTF=
CONFIG_INFINIBAND_ON_DEMAND_PAGING=y
CONFIG_BF_DEVICE_EMULATION=
CONFIG_BF_POWER_FAILURE_EVENT=
CONFIG_GPU_DIRECT_STORAGE=
CONFIG_ENABLE_MLX5_FS_DEBUGFS=
LINUXINCLUDE='
-D__OFED_BUILD__
-D__KERNEL__
-DCOMPAT_BASE=""mlnx-ofa_kernel-compat-20201231-1804-13e6dd1"" -DCOMPAT_BASE_TREE=""mlnx_ofed/mlnx-ofa_kernel-4.0.git"" -DCOMPAT_BASE_TREE_VERSION=""13e6dd1"" -DCOMPAT_PROJECT=""Compat-mlnx-ofed"" -DCOMPAT_VERSION=""13e6dd1""
-include /lib/modules/4.18.0-193.3.0.0.el7.x86_64/build/include/generated/autoconf.h
-include /lib/modules/4.18.0-193.3.0.0.el7.x86_64/build/include/linux/kconfig.h
-include /root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default/include/linux/compat-2.6.h



-I/root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default/include
-I/root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default/include/uapi
-I/root/rpmbuild/BUILD/mlnx-ofa_kernel-5.2/obj/default/drivers/infiniband/debug
$(if $(CONFIG_XEN),-D__XEN_INTERFACE_VERSION__=$(CONFIG_XEN_INTERFACE_VERSION))
$(if $(CONFIG_XEN),-I$(srctree)/arch/x86/include/mach-xen)
-I$(srctree)/arch/$(SRCARCH)/include
-Iarch/$(SRCARCH)/include/generated
-Iinclude
-I$(srctree)/arch/$(SRCARCH)/include/uapi
-Iarch/$(SRCARCH)/include/generated/uapi
-I$(srctree)/include
-I$(srctree)/include/uapi
-Iinclude/generated/uapi
$(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include)
-I$(srctree)/arch/$(SRCARCH)/include
-Iarch/$(SRCARCH)/include/generated
'
modules
make[1]: Entering directory '/usr/src/kernels/4.18.0-193.3.0.0.el7.x86_64'
arch/x86/Makefile:184: *** Compiler lacks asm-goto support.. Stop.
make[1]: Leaving directory '/usr/src/kernels/4.18.0-193.3.0.0.el7.x86_64'
make: *** [kernel] Error 2
"

but every thing will fine without the kpatch-cc.

I have verified the CC_HAVE_ASM_GOTO macro defined in the kernel most outside Makefile.

Is there anything I missed to make a OOT livepatch?

@wardenjohn
Copy link
Contributor

It seem the output:

running build
running build_py
running build_scripts

is generated by yourself. What is the build_py and build scripts?
Do you mind providing them?

@Heathcliff4689
Copy link
Author

Heathcliff4689 commented Dec 23, 2024

It seem the output:

running build
running build_py
running build_scripts

is generated by yourself. What is the build_py and build scripts? Do you mind providing them?

of course not.

Although I think it is normal.

These three lines was output by cd ofed_scripts/utils && python setup.py build && cd -

the setup.py is

from distutils.core import setup
from subprocess import Popen, PIPE
import sys
from sys import argv

# We cannot sanely support python 2.4:
if sys.version_info < (2, 5):
    print('requires python 2.5 and up')
    sys.exit(0)

# I would absolutely *LOVE* to be informed of a sexier way to do this,
# preferably without hard-coding Ubuntu as a special case...
try:
    if 'Ubuntu\n' in Popen(('lsb_release', '-si'),
            stdout=PIPE).communicate():
        argv.append('--install-layout=deb')
except OSError:
    pass

setup(name='ofed-le-utils',
      version='1.0.3',
      author='Amir Vadai',
      author_email='[email protected]',
      url='www.mellanox.co.il',
      scripts=['mlnx_qos', 'tc_wrap.py', 'mlnx_perf', 'mlnx_dump_parser', 'mlx_fs_dump'],
      py_modules=['netlink', 'dcbnetlink', 'genetlink'],
      )

and the total related file can be found from Nvidia (I have attached the src rpm here).
mlnx-ofa_kernel-5.2-OFED.5.2.1.0.4.1.tar.gz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants