Make the pidfile accessible by everyone

[mcasquer]

Creates the pidfile at an accessible location
for every user, this way the manage pidfile
warning is avoided.

Pull Request Checklist

• implement the feature
• write the documentation
• extend the test coverage
• update the specification
• mention the version
• include a release note

[mcasquer]

To discuss here which of the added # noqa could be avoided

[mcasquer]

@happz one question, in the related issue with the patch it's used tmt -a provision -h local which IIUC it's the local host already executing tmt, in this case the code raises again permissions issues when doing the chmod

[happz]

@happz one question, in the related issue with the patch it's used tmt -a provision -h local which IIUC it's the local host already executing tmt, in this case the code raises again permissions issues when doing the chmod

Is your change here causing a regression? Or is your change here just making no difference WRT #2877?

[happz]

This one's easy: as an unprivileged user, you cannot create /var/tmp/tmt/pid directory:

Correct me if I'm wrong, but adding sudo to the tmt commands would be fine in this case?

I believe so. Check guest fact whether superuser is enabled, from the setup() method it should be something like self.facts.is_superuser, and if it's false, we will need to run the script with sudo. It should be possible to incorporate it into the ShellScript content.

[happz]

Hmmm, I think we need to be more strict: I believe tmt running in TF sets these for our tests, to the old value of path and directory, and since we allow the variable to exist, the old ones sneak in and ruin the test. SO we need the following change, at least:

        rlRun "TMT_TEST_PIDFILE_ROOT=/var/tmp/tmt/pid"
        rlRun "TMT_TEST_PIDFILE=$TMT_TEST_PIDFILE_ROOT/tmt-test.pid"
        rlRun "TMT_TEST_PIDFILE_LOCK=$TMT_TEST_PIDFILE.lock"

See the log:

:: [ 13:02:45 ] :: [  BEGIN   ] :: Running 'TMT_TEST_PIDFILE_ROOT=/var/tmp/tmt/pid'
:: [ 13:02:45 ] :: [   PASS   ] :: Command 'TMT_TEST_PIDFILE_ROOT=/var/tmp/tmt/pid' (Expected 0, got 0)
:: [ 13:02:45 ] :: [  BEGIN   ] :: Running 'TMT_TEST_PIDFILE=/var/tmp/tmt-test.pid'
:: [ 13:02:45 ] :: [   PASS   ] :: Command 'TMT_TEST_PIDFILE=/var/tmp/tmt-test.pid' (Expected 0, got 0)
:: [ 13:02:45 ] :: [  BEGIN   ] :: Running 'TMT_TEST_PIDFILE_LOCK=/var/tmp/tmt-test.pid.lock'
:: [ 13:02:45 ] :: [   PASS   ] :: Command 'TMT_TEST_PIDFILE_LOCK=/var/tmp/tmt-test.pid.lock' (Expected 0, got 0)

File locations are obviously incorrect, old.

[happz]

@mcasquer I'm afraid running test cases - tmt --context provisiontest=$PROVISION_HOW run -rvvv plan --name /test/root - with sudo will hide the problem: tmt, when told to use a less-privileged account to run tests on a guest, will fail to create pidfile directory. Adding sudo to the test will not remove the problem, the test may pass, but the problem will remain. IIUIC, the solutions should be adding sudo to the command actually creating the pidfile directory, i.e. https://github.com/teemtee/tmt/pull/3252/files#diff-f684593942660856c3bdc810b597c3956b4107716a554eece2c2bf08344a9503R928.

Check guest fact whether superuser is enabled, from the setup() method it should be something like self.facts.is_superuser, and if it's false, we will need to run the script with sudo. It should be possible to incorporate it into the ShellScript content.

        sudo = 'sudo' if not self.facts.is_superuser and self.become else ''

        self.execute(ShellScript(
            f"""
            if [ ! -d {pid_directory} ]; then \
                   {sudo} mkdir -p {pid_directory} \
                && {sudo} chmod ugo+rwx {pid_directory}; \
            fi
            """
            ))

See a similar problem solved with the same approach: https://github.com/teemtee/tmt/blob/main/tmt/steps/provision/__init__.py#L1714. That should teach tmt to use sudo when needed for privileged operations.

[happz]

One thing I noticed: there's Guest.setup() and GuestSsh.setup(). The former is pretty empty, and you are adding first code into it; the latter already contains some code.

Byw GuestSsh.setup() does not call its base class implementation, it lacks super().setup() call. Looks like an omission which was fine, Guest.setup() did nothing so it didn't hurt. Now it does, and we need to call it. Please, add the super() call to GuestSsh so we have the chain complete.

[mcasquer]

So with this you mean it'd better if I move the code from Guest.setup() to the GuestSsh one and add the super().setup() ? or only do the last part in the Guest.setup() function?

[happz]

Nono, I think Guest is the right place. We just need one extra call at the beginning of GuestSsh.setup, and that would be calling its base class implementation of the method it overrides.

[mcasquer]

Got it, code updated, thanks!

[mcasquer]

IIUIC, the solutions should be adding sudo to the command actually creating the pidfile directory,

Oh, sorry for that I misunderstood your comment above, thanks a lot for so detailed explanation

[happz]

@mcasquer my mistake: drop the self.become part of the condition, it must apply every time user is not a superuser:

diff --git a/tmt/steps/provision/__init__.py b/tmt/steps/provision/__init__.py
index 907d17dc..7dcda9d3 100644
--- a/tmt/steps/provision/__init__.py
+++ b/tmt/steps/provision/__init__.py
@@ -924,7 +924,7 @@ class Guest(tmt.utils.Common):
         Setup the guest after it has been started. It is called after :py:meth:`Guest.start`.
         """
         from tmt.steps.execute.internal import effective_pidfile_root
-        sudo = 'sudo' if not self.facts.is_superuser and self.become else ''
+        sudo = 'sudo' if not self.facts.is_superuser else ''
         pid_directory = effective_pidfile_root()
         self.execute(ShellScript(
             f"""

That should resolve the test finally :)

[mcasquer]

That should resolve the test finally :)

Updated !

[psss]

Summary from the chat discussion: /tests/provision/facts/guest should be adjusted to use the container image with password-less sudo that is localhost/tmt/tests/container/fedora/39/unprivileged and we will explicitly document that one of the following is needed on the guest:

• root user (with all permissions)
• regular user (with password-less sudo configured)

[happz]

--user would need to change as well, to fedora, and the main image shouldn't be the unprivileged one. Together, I think something like this should be the outcome:

provision_options="--image $TEST_IMAGE_PREFIX/fedora/39:latest"
bfu_provision_options="--image $TEST_IMAGE_PREFIX/fedora/39/unprivileged:latest --user=fedora"

[mcasquer]

It seems now there's an issue when setting the ACLs, the user doesn't have permissions

cmd:
            mkdir -p /var/tmp/tmt;
            setfacl -d -m o:rX /var/tmp/tmt
            err: setfacl: /var/tmp/tmt: Operation not permitted

Also I see some extra error about an rsync command along with the following message
fail: Failed to push workdir to the guest. This usually means that login as 'fedora' to the guest does not work.

[martinhoyer]

lgtm, thanks!

[mcasquer]

I see now the failed tests are
/tests/core/environment-file

:: [ 10:30:11 ] :: [   FAIL   ] :: Command 'tmt run --environment STR=bad_str -rvvvddd plan --name /plan/good 2>&1             | tee output' (Expected 1, got 2)
:: [ 10:30:11 ] :: [   FAIL   ] :: File 'output' should contain 'AssertionError: assert 'bad_str' == 'O'' 

/tests/prepare/ansible

:: [ 10:54:20 ] :: [   FAIL   ] :: Command 'tmt run -i /tmp/tmp.7zaDNljweN --scratch -av provision -h container -i localhost/tmt/tests/container/alpine:latest plan -n /remote' (Expected 0, got 2)
:: [ 10:54:20 ] :: [   FAIL   ] :: File '/tmp/tmp.7zaDNljweN/log.txt' should contain 'ansible-playbook -vvv' 

Sorry I don't see how is this related with the patch... 😕 @happz @psss could you shed some light here?

[psss]

Hmmm, these seem to be caused by GitHub randomly refusing to serve the raw content:

fail: Failed to fetch remote playbook 'https://raw.githubusercontent.com/teemtee/tmt/main/tests/prepare/ansible/data/playbook.yml'.
403 Client Error: Forbidden for url: https://raw.githubusercontent.com/teemtee/tmt/22a46a4a6760517e3eadbbff0c9bebdb95442760/tests/core/env/data/vars.yaml

Sounds like they enabled some rate limiting, or we hit the limit:

• https://stackoverflow.com/questions/66522261/does-github-rate-limit-access-to-public-raw-files

Any suggestions?