feat: implement new cli command to read tests #986

DudaGod · 2024-08-05T06:48:32Z

What is done

Implement ability to read tests using cli command npx testplane list-tests which support options like for run tests:

--config;
--browser;
--set;
--require;
--grep;
and new options:
--ignore;
--silent;
--output-file;
--formatter.

The formatter supports 2 values: list (default) and tree. When using list user get plain list of unique tests, example:

[
    {
        "id": "5a105e8",
        "titlePath": [
            "test1"
        ],
        "browserIds": [
            "yandex",
            "chrome"
        ],
        "file": "tests/second.hermione.js",
        "pending": false,
        "skipReason": ""
    },
    {
        "id": "d2b3179",
        "titlePath": [
            "suite",
            "test2"
        ],
        "browserIds": [
            "yandex",
            "chrome"
        ],
        "file": "tests/second.hermione.js",
        "pending": false,
        "skipReason": ""
    }
]

The same data in tree formatter looks like:

[
    {
        "id": "36749990",
        "title": "suite",
        "line": 3,
        "column": 1,
        "file": "example.hermione.js",
        "suites": [],
        "tests": [
            {
                "id": "d2b3179",
                "title": "test2",
                "line": 4,
                "column": 5,
                "browserIds": [
                    "yandex"
                ],
                "pending": false,
                "skipReason": ""
            }
        ],
        "pending": false,
        "skipReason": ""
    },
    {
        "id": "5a105e8",
        "title": "test1",
        "line": 1,
        "column": 1,
        "browserIds": [
            "yandex"
        ],
        "file": "example.hermione.js",
        "pending": false,
        "skipReason": ""
    }
]

the important difference is that the tree formatter supports line and column fields. They can be used to find out exactly where the suite or test is declared in the file. It will be used in vscode extension.

The same output is supported via API - testCollection.format(<type>)

DudaGod · 2024-08-05T21:47:15Z

docs/cli.md

- [Reporters](#reporters)
- [Require modules](#require-modules)
+- [`testplane` command](#testplane-command)
+  - [Options](#options)


at the moment, I have described in detail only the options about which there has already been an information

DudaGod · 2024-08-05T21:48:40Z

docs/cli.md

+- `browserIds` (`String[]`) - list of browsers in which the test will be launched;
+- `file` (`String`, only in tests without parent suites) - path to the file relative to the working directory.
+
+### Overriding settings


Nothing has changed here

DudaGod · 2024-08-05T21:49:16Z

docs/component-testing.md

+    - [Logs from the browser console in the terminal](#logs-from-the-browser-console-in-the-terminal)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+


Just run npm toc

DudaGod · 2024-08-05T21:51:38Z

docs/programmatic-api.md

@@ -255,6 +255,9 @@ await testplane.readTests(testPaths, options);
    * **ignore** (optional) `String|Glob|Array<String|Glob>` - patterns to exclude paths from the test search.
    * **sets** (optional) `String[]`– Sets to run tests in.
    * **grep** (optional) `RegExp` – Pattern that defines which tests to run.
+    * **replMode** (optional) `{enabled: boolean; beforeTest: boolean; onFail: boolean;}` - [Test development mode using REPL](./cli.md#--repl). When reading the tests, it checks that only one test is running in one browser.


it was not described earlier, so I added

DudaGod · 2024-08-05T21:53:03Z

docs/writing-tests.md

@@ -8,6 +8,7 @@
 - [Hooks](#hooks)
 - [Skip](#skip)
 - [Only](#only)
+- [Also](#also)


added by script npm run toc

DudaGod · 2024-08-05T22:12:40Z

src/test-reader/mocha-reader/index.js

+                }
+
+                if (!config.ui || config.ui === "bdd") {
+                    const pendingMethodName = `x${methodName}`;


In bdd interface support xdescribe and xit to skip suites/tests

DudaGod · 2024-08-05T22:14:17Z

src/test-reader/mocha-reader/index.js

+        const origStackTraceLimit = Error.stackTraceLimit;
+        const origPrepareStackTrace = Error.prepareStackTrace;
+
+        Error.stackTraceLimit = 2;


Use only 2 last messages from stack trace. First message it is this file. And second test file.

DudaGod · 2024-08-05T22:14:36Z

src/test-reader/mocha-reader/index.js

+
+        const location = obj.stack;
+        Error.stackTraceLimit = origStackTraceLimit;
+        Error.prepareStackTrace = origPrepareStackTrace;


return previous values

DudaGod · 2024-08-05T22:15:01Z

src/test-reader/mocha-reader/index.js

+        Error.stackTraceLimit = origStackTraceLimit;
+        Error.prepareStackTrace = origPrepareStackTrace;
+
+        inBus.once(eventName, runnable => {


subscribe one on add suite or test in order to save location to it

src/utils/cli.ts

DudaGod · 2024-09-03T15:02:26Z

docs/programmatic-api.md

@@ -255,6 +255,9 @@ await testplane.readTests(testPaths, options);
    * **ignore** (optional) `String|Glob|Array<String|Glob>` - patterns to exclude paths from the test search.
    * **sets** (optional) `String[]`– Sets to run tests in.
    * **grep** (optional) `RegExp` – Pattern that defines which tests to run.
+    * **replMode** (optional) `{enabled: boolean; beforeTest: boolean; onFail: boolean;}` - [Test development mode using REPL](./cli.md#--repl). When reading the tests, it checks that only one test is running in one browser.
+    * **runnableOpts** (optional):


new option which gives me ability to save test locations in file (line and column)

DudaGod · 2024-09-03T15:03:30Z

docs/programmatic-api.md

@@ -317,6 +320,8 @@ TestCollection API:

 * `eachRootSuite((root, browserId) => ...)` - iterates over all root suites in collection which have some tests.

+* `format(formatterType)` - formats the tests in one of the available formatting types (`list` or `tree`). You can read more about the available formatting types [here](./cli.md#--formattername).


User can format test not only via cli command but also using testCollection.format method

DudaGod · 2024-09-03T15:17:11Z

src/test-collection/formatters/list.ts

+    const allTestsById = new Map<string, FormatterListTest>();
+
+    testCollection.eachTest((test, browserId) => {
+        if ((test as TestDisabled).disabled) {


should do nothing with disabled tests (for example for passive browsers)

DudaGod · 2024-09-03T15:17:37Z

src/test-collection/formatters/list.ts

+        }
+
+        if (allTestsById.has(test.id)) {
+            const foundTest = allTestsById.get(test.id)!;


for the same test in different browsers, I specify only the field browserIds. In order not to store too much of the same data (except for the browser name)

DudaGod · 2024-09-03T15:20:21Z

src/test-collection/formatters/tree.ts

+        const treeTest = createTreeTest(test, browserId);
+        allTestsById.set(treeTest.id, treeTest);
+
+        collectSuites(test.parent!, treeTest, allSuitesById);


here, in order to create the right tree, I move from the tests (which can be disabled) up the parent suites.

DudaGod · 2024-09-03T15:20:41Z

src/test-collection/formatters/tree.ts

+            const treeTest = allTestsById.get(test.id)!;
+
+            if (!treeTest.browserIds.includes(browserId)) {
+                treeTest.browserIds.push(browserId);


Like for list tests I just collect all browser ids for the same test

src/test-collection/formatters/tree.ts

shadowusr

Great addition overall! 🔥

Looks good to me besides documentation bits. Didn't look at unit tests.

docs/cli.md

docs/component-testing.md

src/cli/commands/list-tests/index.ts

src/test-collection/formatters/list.ts

src/test-collection/index.ts

src/utils/cli.ts

src/test-reader/mocha-reader/utils.js

src/test-collection/formatters/tree.ts

DudaGod force-pushed the HERMIONE-8.cli_cmd_to_read_tests branch from 4aa6d14 to c160044 Compare August 5, 2024 15:47

DudaGod commented Aug 5, 2024

View reviewed changes

DudaGod force-pushed the HERMIONE-8.cli_cmd_to_read_tests branch 2 times, most recently from 3d6e420 to caaa36d Compare August 12, 2024 23:50

DudaGod force-pushed the HERMIONE-8.cli_cmd_to_read_tests branch 2 times, most recently from b9a8810 to 4087fd7 Compare September 3, 2024 15:15

DudaGod commented Sep 3, 2024

View reviewed changes

shadowusr approved these changes Sep 4, 2024

View reviewed changes

DudaGod force-pushed the HERMIONE-8.cli_cmd_to_read_tests branch from 4087fd7 to 3d47d40 Compare September 9, 2024 21:01

feat: implement new cli command to read tests

ec7a5e8

DudaGod force-pushed the HERMIONE-8.cli_cmd_to_read_tests branch from 3d47d40 to ec7a5e8 Compare September 9, 2024 21:02

DudaGod merged commit aad3322 into master Sep 9, 2024
2 checks passed

DudaGod deleted the HERMIONE-8.cli_cmd_to_read_tests branch September 9, 2024 21:58

DudaGod mentioned this pull request Sep 10, 2024

docs(cli): add info about new command: "list-tests" gemini-testing/testplane-docs#36

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement new cli command to read tests #986

feat: implement new cli command to read tests #986

DudaGod commented Aug 5, 2024 •

edited

Loading

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Aug 5, 2024

DudaGod Sep 3, 2024

DudaGod Sep 3, 2024

DudaGod Sep 3, 2024

DudaGod Sep 3, 2024

DudaGod Sep 3, 2024

DudaGod Sep 3, 2024

shadowusr left a comment

		- [Logs from the browser console in the terminal](#logs-from-the-browser-console-in-the-terminal)

		<!-- END doctoc generated TOC please keep comment here to allow auto update -->

		@@ -317,6 +320,8 @@ TestCollection API:

		* `eachRootSuite((root, browserId) => ...)` - iterates over all root suites in collection which have some tests.

		* `format(formatterType)` - formats the tests in one of the available formatting types (`list` or `tree`). You can read more about the available formatting types [here](./cli.md#--formattername).

feat: implement new cli command to read tests #986

feat: implement new cli command to read tests #986

Conversation

DudaGod commented Aug 5, 2024 • edited Loading

What is done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shadowusr left a comment

Choose a reason for hiding this comment

DudaGod commented Aug 5, 2024 •

edited

Loading