Beakerlib tests report "/usr/local/bin/tmt-report-result: No such file or directory"

[imphil]

Since release tmt 1.38.0 I'm getting an error when running beakerlib tests locally. I'm installing tmt through pip.

out: /usr/share/beakerlib/testing.sh: line 1076: /usr/local/bin/tmt-report-result: No such file or directory
out: :: [ 14:05:34 ] :: [  ERROR   ] :: rlReport: Failed to report the result

...

out: /usr/share/beakerlib/journal.sh: line 270: /usr/local/bin/tmt-file-submit: No such file or directory
out: :: [ 14:05:42 ] :: [  ERROR   ] :: rlJournalEnd: Submit wasn't successful

The file is not there; it's in /home/vscode/.local/lib/python3.9/site-packages/tmt/steps/execute/scripts/tmt-report-result in my case.

I can make the error go away if I manually set TMT_SCRIPTS_DIR, like so:

export TMT_SCRIPTS_DIR=/home/vscode/.local/lib/python3.9/site-packages/tmt/steps/execute/scripts 

I'm not quite sure what the expected behavior is supposed to be. Should I be setting TMT_SCRIPTS_DIR manually? Should tmt set that, based on where it knows pip placed the scripts? Should the scripts be a Python entry point and hence copied into ~/.local/bin by pip? Should the local provisioner do something? I'm happy to do a PR once I know where this should be going.

Bisect

I'm assuming this change of behavior was introduced in #3200, for which the release notes mention:

Support for RHEL-like operating systems in Image Mode has been added. The destination directory of the scripts added by tmt on these operating systems is /var/lib/tmt/scripts. For all others the /usr/local/bin destination directory is used. A new environment variable TMT_SCRIPTS_DIR is available to override the default locations.

Steps to reproduce

# Run tmt locally
tmt run --all -vvv plan --name /packages/my-pkg/plans/91.0.0 provision --how local --feeling-safe

Test log (a bit reduced)

    execute
        queued execute task #1: default-0 on default-0
        
        execute task #1: default-0 on default-0
        how: tmt
        order: 50
        exit-first: false
            test: Smoke test
                cmd: ./test.sh
...
                out: /usr/share/beakerlib/testing.sh: line 1076: /usr/local/bin/tmt-report-result: No such file or directory
                out: :: [ 14:05:34 ] :: [  ERROR   ] :: rlReport: Failed to report the result
                out: 
                out: ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
                out: ::   Test: my-test
                out: ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
...
                out: :: [ 14:05:37 ] :: [   PASS   ] :: Call without arguments (Expected 0, got 0)
                out: :: [ 14:05:37 ] :: [   PASS   ] :: File 'my.out' should contain 'are missing from the command line' 
                out: ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
                out: ::   Duration: 1s
                out: ::   Assertions: 2 good, 0 bad
                out: ::   RESULT: PASS (Test: my-test)
                out: 
                out: /usr/share/beakerlib/testing.sh: line 1076: /usr/local/bin/tmt-report-result: No such file or directory
                out: :: [ 14:05:37 ] :: [  ERROR   ] :: rlReport: Failed to report the result
...
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
                out: 
                out: :: [ 14:05:42 ] :: [   LOG    ] :: Phases fingerprint:  GpTeNtEf
                out: :: [ 14:05:42 ] :: [   LOG    ] :: Asserts fingerprint: GLWAc0zH
                out: /usr/share/beakerlib/journal.sh: line 270: /usr/local/bin/tmt-file-submit: No such file or directory
                out: :: [ 14:05:42 ] :: [  ERROR   ] :: rlJournalEnd: Submit wasn't successful
                out: ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
                out: ::   Duration: 11s
                out: ::   Phases: 3 good, 0 bad
                out: ::   OVERALL RESULT: PASS ()
                out: 
                00:00:13 pass /packages/my-pkg/tests/smoke (on default-0) [1/1]

The test itself is trivial.

#!/bin/bash
. /usr/share/beakerlib/beakerlib.sh || exit 1

rlJournalStart
    rlPhaseStartSetup
        rlRun "tmp=\$(mktemp -d)" 0 "Create tmp directory"
        rlRun "pushd $tmp"
        rlRun "set -o pipefail"
    rlPhaseEnd

    rlPhaseStartTest "Test: my-test"
        rlRun "something 2>&1 | tee my.out" 0 "Call without arguments"
        rlAssertGrep 'something' "my.out"
    rlPhaseEnd

    rlPhaseStartCleanup
        rlRun "popd"
        rlRun "rm -r $tmp" 0 "Remove tmp directory"
    rlPhaseEnd
rlJournalEnd