Duplicate tests getting generated to handle Outline Scenarios

[PavanMudigondaTR]

I started exploring "Playwright-BDD" extension and I see the test cases are getting duplicates for "Scenario Outline" which is not the case in Cucumber.js. Is there a way we can handle this as well in future ? Basically instead of generating duplicate tests, we need to find a way to consume data from parameters grid while executing tests. I do understand this is best option you had for handling Scenario Outline. Is it possible for Playwright Test and Playwright BDD extension to work together to come up with better solution ?

Potential Solution:
Generate in-line csv in to "test.describe" block and make the Tests repeat using the csv ? Is something like that possible using this playwright pattern for csv test repeats ?

Some background why I ask for this:

• We use automation patterns to synchronize tests from Feature Files to Azure DevOps. Outline Feature (lets say I have a test case to test home page of website. I have 50 test data combinations cause we run some of our applications home page in 50 languages with 50 different URLs ). I have only ONE test case in ADO TestPlans with these 50 data combinations
• Next when I run tests..it will test with all parameter combinations given in parameters section. Azure DevOps TesPlans stores test results under one test case.
• When I Synchronize test results...I use Cucumber Json File which updates the test results for all combinations under one test case.
• If I use this extension I need to have 50 test cases created in Azure DevOps and the Testers and Developers will get confused while reading tests as they all look same.

[vitalets]

Playwright itself does not allow duplicate test titles. For example this produces error:

test('test', () => {});
test('test', () => {});

// Error: duplicate test title "test", first declared in test/gen/gen.spec.ts:10

That's why for scenario outline playwright-bdd tests has index in title:

test('test', () => {});
test('test (1)', () => {});

We can wrap it into test.describe and build title from parameters:

test.describe('test', () => {
  test('product=english,title=Home', () => {});
  test('product=xxx,title=YYY', () => {});
});

But this also generates new "test entities" that does not match grouping from cucumber. Maybe it can be solved on reporting level..

When I Synchronize test results...I use Cucumber Json File which updates the test results for all combinations under one test case.

Could you show an example of Cucumber Json File with scenario outline?

[PavanMudigondaTR]

test.describe

thanks for quick response. your solution of "test.describe" seems okay to me. that will solve half problem. I will provide you Cucumber Json report for Outline Tests tomorrow. If we can manipulate and output the Cucumber Json to align with Cucumber.js pattern for Outline Scenarios that will solve 100% of issue.

[vitalets]

Since v2.0.0 Scenario Outline is wrapped into test.describe and each example row is converted to test with synthetic title "Example #{num}". (Although I'm still thinking of solving this on reporting side, see #9). Feel free to share feedback.

Feature:

Feature: scenario-outline

  Scenario Outline: Check doubled
    Then Doubled <start> to equal <end>

  Examples:
    | start | end |
    |    2  |   4 |
    |    3  |   6 |

Test:

test.describe("scenario-outline", () => {

  test.describe("Check doubled", () => {

    test("Example #1", async ({ Given, Then }) => {
      await Then("Doubled 2 to equal 4");
    });

    test("Example #2", async ({ Given, Then }) => {
      await Then("Doubled 3 to equal 6");
    });

  });

});