xdist causes duplicate reports for collector errors

[musateduard]

hi,

when running pytest with --numprocesses=4 (or any non 0 number) if you generate an xml or json report using the json plugin and some of the modules fail during collection, xdist will cause pytest to generate a collector error for the same file for each worker. ex.: 4 processes will cause 4 entries for each failed collection, 2 processes -> 2 entries per failed collection etc.

json example when using --numprocesses=4

    "collectors": [
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        }
    ]

correct output when using --numprocesses=0

    "collectors": [
        {
            "nodeid": "",
            "outcome": "passed",
            "result": [
                {
                    "nodeid": "tests/test_other2.py",
                    "type": "Module"
                }
            ]
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace>"
        }
    ],

also, when running using json report, normally the collectors key should be added for all collected files but when using xdist only failed collections get added.

example with --numprocesses=4 and 2 files: 1 that fails during collection and 1 that doesn't. notice how none of the successful collections show up and the failed collection shows up 4 times.

    "collectors": [
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace message>"
        }
    ]

correct output: when using --numprocesses=0 all collected items are shown including the successful ones and the failed collections only show once per file.

    "collectors": [
        {
            "nodeid": "",
            "outcome": "passed",
            "result": [
                {
                    "nodeid": "tests/test_other.py",
                    "type": "Module"
                },
                {
                    "nodeid": "tests/test_other2.py",
                    "type": "Module"
                }
            ]
        },
        {
            "nodeid": "tests/test_other.py",
            "outcome": "passed",
            "result": [
                {
                    "nodeid": "tests/test_other.py::test_skipped",
                    "type": "Function",
                    "lineno": 3
                },
                {
                    "nodeid": "tests/test_other.py::test_will_pass",
                    "type": "Function",
                    "lineno": 9
                },
                {
                    "nodeid": "tests/test_other.py::test_will_fail",
                    "type": "Function",
                    "lineno": 15
                },
                {
                    "nodeid": "tests/test_other.py::test_will_xfail",
                    "type": "Function",
                    "lineno": 29
                },
                {
                    "nodeid": "tests/test_other.py::test_will_xpass",
                    "type": "Function",
                    "lineno": 36
                },
                {
                    "nodeid": "tests/test_other.py::test_fixture_exception",
                    "type": "Function",
                    "lineno": 43
                },
                {
                    "nodeid": "tests/test_other.py::test_fixture_missing",
                    "type": "Function",
                    "lineno": 49
                }
            ]
        },
        {
            "nodeid": "tests/test_other2.py",
            "outcome": "failed",
            "result": [],
            "longrepr": "<stacktrace>"
        }
    ],

to cause a collection error you can raise an exception at the top of the file or you can import a class that raises an exception inside its __init__ method.

raise Exception("testing expection from module")


def test_import_exception(session_id: str):
    print(session_id)
    assert True
    return

or

from some.folder import InvalidImport


def test_import_exception(session_id: str):
    print(session_id)
    assert True
    return

# some.folder
class InvalidImport:

    raise Exception("testing exception from import")

this affects the pytest xml as well in that the failed collections get printed there multiple times as well, the difference being that xml doesn't have a collectors section, so no successful collections show up.