Skip to content

Commit

Permalink
tests/util: ldd lists files not accessible in scratch container
Browse files Browse the repository at this point in the history
Starting in glibc 2.33, new hwcaps functionality was added which
allows the dynamic linker to load optimized versions of libraries
within a new "glibc-hwcaps" directory in the library search path.

The crio and podman tests use `ldd` to get a list of the shared
libraries dependencies and copies them into a scratch container. On
power9, some of the libraries are symlinked to versioned filenames
of the optimized libraries. This is problematic inside the scratch
container because the binaries are looking for the non-versioned
library name but cannot find it.

For example a snippet of `ldd /usr/bin/ping` shows:
```
	libm.so.6 => /lib64/glibc-hwcaps/power9/libm-2.28.so
```

When the scratch container runs, it cannot find libm.so.6 because
only libm-2.28.so is copied into the scratch container.

I found the removing /etc/ld.so.cache corrects the `ldd` output to
have the same filename as the shared library name allowing the
scratch container to find the shared libraries.

`ldd` output after deleting the cache:

```
	libm.so.6 => /lib64/glibc-hwcaps/power9/libm.so.6
```

This has probably been an issue for a bit now but we were not
building on power9.
  • Loading branch information
mike-nguyen authored and dustymabe committed Oct 27, 2022
1 parent 0086c71 commit 45559e4
Showing 1 changed file with 12 additions and 3 deletions.
15 changes: 12 additions & 3 deletions mantle/kola/tests/util/containers.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,18 @@ import (

// GenPodmanScratchContainer creates a podman scratch container out of binaries from the host
func GenPodmanScratchContainer(c cluster.TestCluster, m platform.Machine, name string, binnames []string) {
// Scratch containers are created by copying a binary and its shared libraries dependencies
// into the container image. `ldd` is used to find the paths to the shared libraries. On
// power9, some shared libraries were symlinked to versioned shared libraries using the
// versioned filename as the target. For example, libm.so.6 would be copied into the scratch
// container as libm-2.28.so. When the versioned shared libraries were copied into the scratch
// container, the dynamic linker could not find the non-versioned filenames. The ld.so.cache
// seemed to have symlinks to the versioned shared libraries. Deleting /etc/ld.so.cache
// restored symlinks to the non-versioned shared libraries.
cmd := `tmpdir=$(mktemp -d); cd $tmpdir; echo -e "FROM scratch\nCOPY . /" > Dockerfile;
b=$(which %s); libs=$(sudo ldd $b | grep -o /lib'[^ ]*' | sort -u);
sudo rsync -av --relative --copy-links $b $libs ./;
sudo podman build --network host --layers=false -t localhost/%s .`
sudo rm -f /etc/ld.so.cache;
b=$(which %s); libs=$(sudo ldd $b | grep -o /lib'[^ ]*' | sort -u);
sudo rsync -av --relative --copy-links $b $libs ./;
sudo podman build --network host --layers=false -t localhost/%s .`
c.RunCmdSyncf(m, cmd, strings.Join(binnames, " "), name)
}

0 comments on commit 45559e4

Please sign in to comment.