AttributeError: 'OpSOLMonitorThread' object has no attribute 'isAlive'

[vpuliyal]

Hi @harihare,

I've installed RHEL9.4 GA on my upstream lpar and try to install upstream kernel on it using op-test. I could see below error while running op-test.

^[[?2004h[console-expect]#[ -d /home/linux_src ] || mkdir -p /home/linux_src
[ -d /home/linux_src ] || mkdir -p /home/linux_src
^[[?2004l^M^[[?2004h[console-expect]#echo $?
echo $?
^[[?2004l^M0
ERROR

======================================================================
ERROR: runTest (testcases.InstallUpstreamKernel.InstallUpstreamKernel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/op-test/testcases/InstallUpstreamKernel.py", line 90, in runTest
    con.run_command("[ -d %s ] || mkdir -p %s" %
  File "/var/lib/jenkins/workspace/op-test/common/OpTestSSH.py", line 225, in run_command
    return self.util.run_command(self, command, timeout, retry)
  File "/var/lib/jenkins/workspace/op-test/common/OpTestUtil.py", line 1685, in run_command
    raise cf
  File "/var/lib/jenkins/workspace/op-test/common/OpTestUtil.py", line 1680, in run_command
    output = self.try_command(term_obj, command, timeout)
  File "/var/lib/jenkins/workspace/op-test/common/OpTestUtil.py", line 1755, in try_command
    raise CommandFailed(command, res, echo_rc)
common.Exceptions.CommandFailed: Command '[ -d /home/linux_src ] || mkdir -p /home/linux_src' exited with '-1'.
Output
['\x1b[?2004l', '\x1b[?2004h']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/op-test/testcases/InstallUpstreamKernel.py", line 175, in runTest
    if self.console_thread.isAlive():
AttributeError: 'OpSOLMonitorThread' object has no attribute 'isAlive'

----------------------------------------------------------------------
Ran 1 test in 181.577s

FAILED (errors=1)

Initially I though that issue with python . So I tried with python3.6, python3.9, python3.10. Still seeing same 'isAlive' issue

isAlive was removed from Python 3.9: https://bugs.python.org/issue37804.

Did I miss anything?

[abdhaleegit]

@vpuliyal the problem is even before that.. it fails on rm /mkdir command... can you try this commands on your linux shell and rerun again

login to linux lpar and run this commands

$ echo "set enable-bracketed-paste off" > .inputrc
$ bind 'set enable-bracketed-paste off'

[vpuliyal]

@abdhaleegit I tried as per your suggestion. But I could see below error.

[console-expect]#echo $?
echo $?
0
ERROR

======================================================================
ERROR: runTest (testcases.InstallUpstreamKernel.InstallUpstreamKernel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/op-test/testcases/InstallUpstreamKernel.py", line 175, in runTest
    if self.console_thread.isAlive():
AttributeError: 'OpSOLMonitorThread' object has no attribute 'isAlive'

----------------------------------------------------------------------
Ran 1 test in 633.143s

FAILED (errors=1)
2024-08-20 12:57:54,710:op-test:<module>:INFO:Exit with Result errors="1" and failures="0"
2024-08-20 12:57:54,769:op-test.common.OpTestUtil:cleanup:INFO:OpTestSystem Starting to Gather ESEL's
2024-08-20 12:57:54,770:op-test.common.OpTestUtil:dump_versions:INFO:Log Location: /var/lib/jenkins/workspace/op-test/test-reports/test-run-20240820124638/*debug*
2024-08-20 12:57:54,770:op-test.common.OpTestUtil:dump_versions:INFO:

[PraveenPenguin]

@vpuliyal are you using the latest copy of op-test?

[vpuliyal]

@PraveenPenguin I'm using https://github.com/open-power/op-test link

[vpuliyal]

@PraveenPenguin can you provide me latest op-test code ?

[sacsant]

@PraveenPenguin can you provide me latest op-test code ?

Are you sure the copy you are using is latest one? The line number from the failure message does not match current code
https://github.com/open-power/op-test/blob/master/testcases/InstallUpstreamKernel.py#L175

Please update your existing local copy via git pull or download it afresh via git clone

[vpuliyal]

@sacsant I used latest op-test code and still seeing below errors.

`[console-expect]#make -j 81 -s && make modules_install && make install
make -j 81 -s && make modules_install && make install
In file included from ^[[01m^[[K./include/asm-generic/bug.h:22^[[m^[[K,
from ^[[01m^[[K./arch/powerpc/include/asm/bug.h:116^[[m^[[K,
from ^[[01m^[[K./include/linux/bug.h:5^[[m^[[K,
from ^[[01m^[[K./include/linux/thread_info.h:13^[[m^[[K,
from ^[[01m^[[K./include/asm-generic/preempt.h:5^[[m^[[K,
from ^[[01m^[[K./arch/powerpc/include/generated/asm/preempt.h:1^[[m^[[K,
from ^[[01m^[[K./include/linux/preempt.h:79^[[m^[[K,
from ^[[01m^[[K./include/linux/spinlock.h:56^[[m^[[K,
from ^[[01m^[[K./include/linux/wait.h:9^[[m^[[K,
from ^[[01m^[[K./include/linux/wait_bit.h:8^[[m^[[K,
from ^[[01m^[[K./include/linux/fs.h:6^[[m^[[K,
from ^[[01m^[[K./include/linux/debugfs.h:15^[[m^[[K,
from ^[[01m^[[Kdrivers/gpu/drm/radeon/r100.c:29^[[m^[[K:
In function '^[[01m^[[Kr100_cp_init_microcode^[[m^[[K',
inlined from '^[[01m^[[Kr100_cp_init^[[m^[[K' at ^[[01m^[[Kdrivers/gpu/drm/radeon/r100.c:1136:7^[[m^[[K:
^[[01m^[[K./include/linux/printk.h:465:44:^[[m^[[K ^[[01;31m^[[Kerror: ^[[m^[[K'^[[01m^[[K%s^[[m^[[K' directive argument is null [^[[01;31m^[[K^[]8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wformat-overflow=^G-Werror=format-overflow=^[]8;;^G^[[m^[[K]
465 | #define printk(fmt, ...) printk_index_wrap(^[[01;31m^[[K_^[[m^[[Kprintk, fmt, ##VA_ARGS)
| ^[[01;31m^[[K^^[[m^[[K
^[[01m^[[K./include/linux/printk.h:437:17:^[[m^[[K ^[[01;36m^[[Knote: ^[[m^[[Kin definition of macro '^[[01m^[[Kprintk_index_wrap^[[m^[[K'
437 | ^[[01;36m^[[K_p_func^[[m^[[K(_fmt, ##VA_ARGS);
| ^[[01;36m^[[K^~~~~~~^[[m^[[K
^[[01m^[[K./include/linux/printk.h:508:9:^[[m^[[K ^[[01;36m^[[Knote: ^[[m^[[Kin expansion of macro '^[[01m^[[Kprintk^[[m^[[K'
508 | ^[[01;36m^[[Kprintk^[[m^[[K(KERN_ERR pr_fmt(fmt), ##VA_ARGS)
| ^[[01;36m^[[K^~~~~~^[[m^[[K
^[[01m^[[Kdrivers/gpu/drm/radeon/r100.c:1062:17:^[[m^[[K ^[[01;36m^[[Knote: ^[[m^[[Kin expansion of macro '^[[01m^[[Kpr_err^[[m^[[K'
1062 | ^[[01;36m^[[Kpr_err^[[m^[[K("radeon_cp: Failed to load firmware "%s"\n", fw_name);
| ^[[01;36m^[[K^~~~~~^[[m^[[K
cc1: all warnings being treated as errors
make[6]: *** [scripts/Makefile.build:244: drivers/gpu/drm/radeon/r100.o] Error 1
make[5]: *** [scripts/Makefile.build:485: drivers/gpu/drm/radeon] Error 2
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [scripts/Makefile.build:485: drivers/gpu/drm] Error 2
make[3]: *** [scripts/Makefile.build:485: drivers/gpu] Error 2
make[2]: *** [scripts/Makefile.build:485: drivers] Error 2
make[1]: *** [/home/linux_src/linux/Makefile:1925: .] Error 2
make: *** [Makefile:224: __sub-make] Error 2
[console-expect]#echo $?
echo $?
2
ERROR

======================================================================
ERROR: runTest (testcases.InstallUpstreamKernel.InstallUpstreamKernel)

Traceback (most recent call last):
File "/var/lib/jenkins/workspace/op-test/testcases/InstallUpstreamKernel.py", line 129, in runTest
con.run_command("make -j %d -s && make modules_install && make install" %
File "/var/lib/jenkins/workspace/op-test/common/OpTestSSH.py", line 225, in run_command
return self.util.run_command(self, command, timeout, retry)
File "/var/lib/jenkins/workspace/op-test/common/OpTestUtil.py", line 1820, in run_command
raise cf
File "/var/lib/jenkins/workspace/op-test/common/OpTestUtil.py", line 1815, in run_command
output = self.try_command(term_obj, command, timeout)
File "/var/lib/jenkins/workspace/op-test/common/OpTestUtil.py", line 1890, in try_command
raise CommandFailed(command, res, echo_rc)
common.Exceptions.CommandFailed: Command 'make -j 81 -s && make modules_install && make install' exited with '2'.
Output
['In file included from \x1b[01m\x1b[K./include/asm-generic/bug.h:22\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./arch/powerpc/include/asm/bug.h:116\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/bug.h:5\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/thread_info.h:13\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/asm-generic/preempt.h:5\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./arch/powerpc/include/generated/asm/preempt.h:1\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/preempt.h:79\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/spinlock.h:56\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/wait.h:9\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/wait_bit.h:8\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/fs.h:6\x1b[m\x1b[K,', ' from \x1b[01m\x1b[K./include/linux/debugfs.h:15\x1b[m\x1b[K,', ' from \x1b[01m\x1b[Kdrivers/gpu/drm/radeon/r100.c:29\x1b[m\x1b[K:', "In function '\x1b[01m\x1b[Kr100_cp_init_microcode\x1b[m\x1b[K',", " inlined from '\x1b[01m\x1b[Kr100_cp_init\x1b[m\x1b[K' at \x1b[01m\x1b[Kdrivers/gpu/drm/radeon/r100.c:1136:7\x1b[m\x1b[K:", "\x1b[01m\x1b[K./include/linux/printk.h:465:44:\x1b[m\x1b[K \x1b[01;31m\x1b[Kerror: \x1b[m\x1b[K'\x1b[01m\x1b[K%s\x1b[m\x1b[K' directive argument is null [\x1b[01;31m\x1b[K\x1b]8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wformat-overflow=\x07-Werror=format-overflow=\x1b]8;;\x07\x1b[m\x1b[K]", ' 465 | #define printk(fmt, ...) printk_index_wrap(\x1b[01;31m\x1b[K_\x1b[m\x1b[Kprintk, fmt, ##VA_ARGS)', ' | \x1b[01;31m\x1b[K^\x1b[m\x1b[K', "\x1b[01m\x1b[K./include/linux/printk.h:437:17:\x1b[m\x1b[K \x1b[01;36m\x1b[Knote: \x1b[m\x1b[Kin definition of macro '\x1b[01m\x1b[Kprintk_index_wrap\x1b[m\x1b[K'", ' 437 | \x1b[01;36m\x1b[K_p_func\x1b[m\x1b[K(_fmt, ##VA_ARGS); \', ' | \x1b[01;36m\x1b[K^~~~~~~\x1b[m\x1b[K', "\x1b[01m\x1b[K./include/linux/printk.h:508:9:\x1b[m\x1b[K \x1b[01;36m\x1b[Knote: \x1b[m\x1b[Kin expansion of macro '\x1b[01m\x1b[Kprintk\x1b[m\x1b[K'", ' 508 | \x1b[01;36m\x1b[Kprintk\x1b[m\x1b[K(KERN_ERR pr_fmt(fmt), ##VA_ARGS)', ' | \x1b[01;36m\x1b[K^~~~~~\x1b[m\x1b[K', "\x1b[01m\x1b[Kdrivers/gpu/drm/radeon/r100.c:1062:17:\x1b[m\x1b[K \x1b[01;36m\x1b[Knote: \x1b[m\x1b[Kin expansion of macro '\x1b[01m\x1b[Kpr_err\x1b[m\x1b[K'", ' 1062 | \x1b[01;36m\x1b[Kpr_err\x1b[m\x1b[K("radeon_cp: Failed to load firmware \"%s\"\n", fw_name);', ' | \x1b[01;36m\x1b[K^~~~~~\x1b[m\x1b[K', 'cc1: all warnings being treated as errors', 'make[6]: *** [scripts/Makefile.build:244: drivers/gpu/drm/radeon/r100.o] Error 1', 'make[5]: *** [scripts/Makefile.build:485: drivers/gpu/drm/radeon] Error 2', 'make[5]: *** Waiting for unfinished jobs....', 'make[4]: *** [scripts/Makefile.build:485: drivers/gpu/drm] Error 2', 'make[3]: *** [scripts/Makefile.build:485: drivers/gpu] Error 2', 'make[2]: *** [scripts/Makefile.build:485: drivers] Error 2', 'make[1]: *** [/home/linux_src/linux/Makefile:1925: .] Error 2', 'make: *** [Makefile:224: __sub-make] Error 2']

---

Ran 1 test in 564.273s

FAILED (errors=1)
2024-09-06 22:50:55,962:op-test::INFO:Exit with Result errors="1" and failures="0"
2024-09-06 22:50:56,036:op-test.common.OpTestUtil:cleanup:INFO:OpTestSystem Starting to Gather ESEL's
2024-09-06 22:50:56,037:op-test.common.OpTestUtil:dump_versions:INFO:Log Location: /var/lib/jenkins/workspace/op-test/test-reports/test-run-20240906223939/debug
2024-09-06 22:50:56,037:op-test.common.OpTestUtil:dump_versions:INFO:

OpTestSystem Firmware Versions Tested
(if flashed things like skiboot.lid, may not be accurate)

Firmware Versions Unavailable

---

`

[vpuliyal]

@sacsant we get Failed to load firmware error in radeon gpu driver code. This is compiler warning being treating as an error. I fixed it was by adding below diff.

diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c

index 0b1e19345f43..170404be09ae 100644
--- a/drivers/gpu/drm/radeon/r100.c
+++ b/drivers/gpu/drm/radeon/r100.c
@@ -1059,7 +1059,11 @@ static int r100_cp_init_microcode(struct radeon_device *rdev)
 
        err = request_firmware(&rdev->me_fw, fw_name, rdev->dev);
        if (err) {
-               pr_err("radeon_cp: Failed to load firmware \"%s\"\n", fw_name);
+               if (fw_name) {
+                       pr_err("radeon_cp: Failed to load firmware \"%s\"\n", fw_name);
+               } else {
+                       pr_err("radeon_cp: Failed to load firmware (null)\n");  
+               }
        } else if (rdev->me_fw->size % 8) {
                pr_err("radeon_cp: Bogus length %zu in firmware \"%s\"\n",
                       rdev->me_fw->size, fw_name);

[sacsant]

@sacsant I used latest op-test code and still seeing below errors.

`[console-expect]#make -j 81 -s && make modules_install && make install make -j 81 -s && make modules_install && make install In file included from ^[[01m^[[K./include/asm-generic/bug.h:22^[[m^[[K, from ^[[01m^[[K./arch/powerpc/include/asm/bug.h:116^[[m^[[K, from ^[[01m^[[K./include/linux/bug.h:5^[[m^[[K, from ^[[01m^[[K./include/linux/thread_info.h:13^[[m^[[K, from ^[[01m^[[K./include/asm-generic/preempt.h:5^[[m^[[K, from ^[[01m^[[K./arch/powerpc/include/generated/asm/preempt.h:1^[[m^[[K, from ^[[01m^[[K./include/linux/preempt.h:79^[[m^[[K, from ^[[01m^[[K./include/linux/spinlock.h:56^[[m^[[K, from ^[[01m^[[K./include/linux/wait.h:9^[[m^[[K, from ^[[01m^[[K./include/linux/wait_bit.h:8^[[m^[[K, from ^[[01m^[[K./include/linux/fs.h:6^[[m^[[K, from ^[[01m^[[K./include/linux/debugfs.h:15^[[m^[[K, from ^[[01m^[[Kdrivers/gpu/drm/radeon/r100.c:29^[[m^[[K: In function '^[[01m^[[Kr100_cp_init_microcode^[[m^[[K', inlined from '^[[01m^[[Kr100_cp_init^[[m^[[K' at ^[[01m^[[Kdrivers/gpu/drm/radeon/r100.c:1136:7^[[m^[[K: ^[[01m^[[K./include/linux/printk.h:465:44:^[[m^[[K ^[[01;31m^[[Kerror: ^[[m^[[K'^[[01m^[[K%s^[[m^[[K' directive argument is null [^[[01;31m^[[K^[]8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wformat-overflow=^G-Werror=format-overflow=^[]8;;^G^[[m^[[K] 465 | #define printk(fmt, ...) printk_index_wrap(^[[01;31m^[[K_^[[m^[[Kprintk, fmt, ##VA_ARGS) | ^[[01;31m^[[K^^[[m^[[K ^[[01m^[[K./include/linux/printk.h:437:17:^[[m^[[K ^[[01;36m^[[Knote: ^[[m^[[Kin definition of macro '^[[01m^[[Kprintk_index_wrap^[[m^[[K' 437 | ^[[01;36m^[[K_p_func^[[m^[[K(_fmt, ##VA_ARGS); | ^[[01;36m^[[K^~~~~~~^[[m^[[K ^[[01m^[[K./include/linux/printk.h:508:9:^[[m^[[K ^[[01;36m^[[Knote: ^[[m^[[Kin expansion of macro '^[[01m^[[Kprintk^[[m^[[K' 508 | ^[[01;36m^[[Kprintk^[[m^[[K(KERN_ERR pr_fmt(fmt), ##VA_ARGS) | ^[[01;36m^[[K^~~~~~^[[m^[[K ^[[01m^[[Kdrivers/gpu/drm/radeon/r100.c:1062:17:^[[m^[[K ^[[01;36m^[[Knote: ^[[m^[[Kin expansion of macro '^[[01m^[[Kpr_err^[[m^[[K' 1062 | ^[[01;36m^[[Kpr_err^[[m^[[K("radeon_cp: Failed to load firmware "%s"\n", fw_name); | ^[[01;36m^[[K^~~~~~^[[m^[[K cc1: all warnings being treated as errors make[6]: *** [scripts/Makefile.build:244: drivers/gpu/drm/radeon/r100.o] Error 1 make[5]: *** [scripts/Makefile.build:485: drivers/gpu/drm/radeon] Error 2 make[5]: *** Waiting for unfinished jobs.... make[4]: ***

This is a different issue. The failure is related to kernel build and is not an op-test issue.

[sacsant]

@sacsant we get Failed to load firmware error in radeon gpu driver code. This is compiler warning being treating as an error. I fixed it was by adding below diff.

diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c

index 0b1e19345f43..170404be09ae 100644
--- a/drivers/gpu/drm/radeon/r100.c
+++ b/drivers/gpu/drm/radeon/r100.c
@@ -1059,7 +1059,11 @@ static int r100_cp_init_microcode(struct radeon_device *rdev)
 
        err = request_firmware(&rdev->me_fw, fw_name, rdev->dev);
        if (err) {
-               pr_err("radeon_cp: Failed to load firmware \"%s\"\n", fw_name);
+               if (fw_name) {
+                       pr_err("radeon_cp: Failed to load firmware \"%s\"\n", fw_name);
+               } else {
+                       pr_err("radeon_cp: Failed to load firmware (null)\n");  
+               }
        } else if (rdev->me_fw->size % 8) {
                pr_err("radeon_cp: Bogus length %zu in firmware \"%s\"\n",
                       rdev->me_fw->size, fw_name);

Okay, yeah please report this issue to Linux kernel mailing list with the suggested fix.