Is there a way to select which subtest to run?

[BowenBao]

Like what pytest -k <pattern> does, but on subtest level.

[nicoddemus]

Hi @BowenBao,

This is not supported today, also I'm not sure how it could be made to work, given the user code that drives the subtests that are executed.

[andre-qumulo]

@nicoddemus could you provide a little more context on this?

I'm wondering if there's a way to pass an external value into a pytest test that would then control is the subtest is ran?

My line of thought from your comment is that we could do something along the lines of:

class Tests(unittest.TestCase):
    def test(self) -> None:
          if 'subtest1' in tests_to_run:
               with self.subtest('subtest1'):
                      self.subtest_1()
          if 'subtest2' in tests_to_run:
               with self.subtest('subtest2'):
                      self.subtest_2()

I can't think of how we'd pass in tests_to_run to the unit test outside of an environment variable.

[RonnyPfannschmidt]

This is outside of the scope of subtests

[andre-qumulo]

@RonnyPfannschmidt Ok, could you at least point me in the right direction? It seems like by using subtests we're losing a valuable developer feature from pytest.

[RonnyPfannschmidt]

Subtests are not tests, they are part of a test

Nobody ever proposed/implemented a safe mechanism for selecting them and

[andre-qumulo]

Ok, that still wasn't my question. My question was how do we pass external information to pytests. Do you happen to have a direction to suggest?

[nicoddemus]

@andre-qumulo depends on where the information comes from... do you mean from the command-line? Perhaps a concrete example of how you want to pass that information would be helpful.

[andre-qumulo]

@nicoddemus my challenge is that I'm not familiar enough with how information is typically passed to unit tests outside of environment variables

[nicoddemus]

Sorry, I was not clear: I'm not asking you how you think it should be implemented, rather asking you how you would like to "use" this feature.

For example you posted:

 if 'subtest1' in tests_to_run:
     with self.subtest('subtest1'):
            self.subtest_1()

That's OK, but how do you, as a user, want that to pass that subtest1 downstream so the test can see it.

[andre-qumulo]

That's the part I'm unclear on personally. I don't see a clear way to pass in additional custom arguments via the pytest commandline: https://docs.pytest.org/en/6.2.x/usage.html#getting-help-on-version-option-names-environment-variables I am curious if you know of any ways.

[nicoddemus]

Ahh I see, yes, see here: https://docs.pytest.org/en/latest/example/simple.html#pass-different-values-to-a-test-function-depending-on-command-line-options