RHCOS 4.12: ext.config.shared tests are not detected/run anymore

[travier]

Bug Report

See openshift/os#1033.

Kola does not detect / run shared tests anymore:

$ cosa kola list | grep shared
empty
$ export COREOS_ASSEMBLER_CONTAINER=quay.io/coreos-assembler/coreos-assembler:rhcos-4.11
$ cosa kola list | grep shared
...
ext.config.shared.networking.resolv.systemd-resolved                            [all]                                                           [all]         [fcos]
ext.config.shared.podman.rootless-systemd                                       [qemu-unpriv]                                                   [all]         [all]
...

Environment

What operating system is being used to run coreos-assembler?

Fedora 36

What operating system is being assembled?

RHCOS

Is coreos-assembler running in Podman or Docker?

podman

If Podman, is coreos-assembler running privileged or unprivileged?

Privileged

[travier]

Found issue in c7fb302 from #3082

[travier]

Confirmed that reverting this commit brings back shared tests.

[travier]

Reverted in #3137. I'm working on a re-revert with a fix.

[dustymabe]

Reverted in #3137. I'm working on a re-revert with a fix.

@travier - any updates on ^^ ?

[travier]

I had started working on it before going the revert route but it's a tricky code path and could not get a fix ready at the time. Will have to take a look again or let someone else pick it up.

[travier]

Pushed my non-working draft in #3204

[jbtrystram]

I am trying to investigate that but I can't reproduce the issue.
Since the change was reverted i expect to see the shared tests when building kola from main (3ac75dc):

[coreos-assembler]$ ./mantle/build kola
Building kola
[coreos-assembler]$ 
[coreos-assembler]$ ./bin/kola version
mantle/kola version 0.15.0+296-g3ac75dcf8
[coreos-assembler]$ ./bin/kola list | grep shared
empty

Is this is because i am using a different version of the cosa container ?

[travier]

Where are you running kola? To see the shared tests, you need to run it in a the folder where you ran cosa init for openshift/os: https://github.com/openshift/os/blob/master/docs/development-scos.md

[travier]

Hum, I can't get them either. Maybe we already regressed on this but this is weird.

[jbtrystram]

Where are you running kola? To see the shared tests, you need to run it in a the folder where you ran cosa init for openshift/os: openshift/os@master/docs/development-scos.md

I was simply replicating what you posted in the initial issue above

[travier]

Oh indeed. This is not explicitly written but the current directory for those commands was the folder you get after a cosa init.

[jbtrystram]

Where are you running kola? To see the shared tests, you need to run it in a the folder where you ran cosa init for openshift/os: openshift/os@master/docs/development-scos.md

yeah I tried that, and did not get them, that's why I was confused and commented :)

[travier]

With an image built (i.e. after a cosa fetch && cosa build), I get the ext.config.shared.* tests from cosa kola list | grep shared.

[jbtrystram]

This was probably fixed in #3341
See #3585 for more details