Fail after fixture teardown

[mxkyb]

I am using pytest to run tests which produce some side effects. A fixture is used to allow the test to control specifics of how those side effects are created, and in the teardown of the fixture there is some evaluation if the tests produced correct results.

However if I throw an error in the fixture teardown, pytest marks the associated test as passed with an error. This produces problems in the evaluation, because the test technically should have failed.

I've been trying to use hooks to solve that, but had no success:

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
    outcome = yield
    report = outcome.get_result()
    pytest.fail("fail the test")

With this approach I don't have information about teardown yet.

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()

    if call.when == 'teardown' and call.excinfo:
        # doesn't work anymore
        pytest.fail("fail the test")

With this approach I am already in teardown and can't call pytest.fail anymore.

Is there a good solution for this? I'd like to not move my check logic into a pytest_runtest_call hook, because it relies on request and pytestconfig which I get very simply as arguments to my fixture.

[webknjaz]

Are you looking for pytest_runtest_logreport()?

[mxkyb]

I am not sure! – I am very new to pytest. This hook allows to listen in to reports, if I understand it correctly.
But I still have the problem that pytest reports ============ 1 passed, 1 error in 12.85s ============ when the test failed in teardown. It is confusing that the teardown is not considered to be part of the test

[webknjaz]

I recommend trying to do something similar to this https://github.com/pytest-dev/pytest-forked/blob/b274232/src/pytest_forked/__init__.py#L112. Wherever you can access a TestReport() instance, set its outcome attribute to 'failed': https://docs.pytest.org/en/stable/reference/reference.html#pytest.TestReport.outcome.

Other methods, that crash the hook execution itself, will report that said stage (setup/teardown) crashed.