Logs before rerun appear in html report

[orgads]

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
seq_num = 0

def seq():
  global seq_num
  seq_num += 1
  return seq_num

def test_fail():
  logger.info(f'Test log {seq()}')
  assert False

pytest --reruns 1 --html=report.html

The report has logs of both runs:

def test_fail():
      logger.info(f'Test log {seq()}')
>     assert False
E     assert False

test_my.py:17: AssertionError

------------------------------ Captured log call -------------------------------
INFO     test_my:test_my.py:16 Test log 1


------------------------------ Captured log call -------------------------------
INFO     test_my:test_my.py:16 Test log 2

It looks like there is logic in _process_logs to skip rerun logs, but this function is not called (maybe it was changed in pytest).

    # Don't add captured output to reruns
    if report.outcome != "rerun":

[BeyondEvil]

What's the expectation here? 🤔

Shouldn't the final result have all logs? Including those of reruns?

Maybe it looks like it doesn't on a trivial example, but in the real world the logs might contain crucial information to solve flaky logic.

[orgads]

From my point of view, only the logs of the last (re)run matter.

[BeyondEvil]

Ok, maybe that makes sense if the logs for previous runs are available in those runs in the report.

Can you clarify what you mean when you say that "_process_logs" isn't called?

@orgads

[orgads]

I didn't say it isn't called, maybe it's called earlier, or maybe not in the right context, or maybe I misunderstood its purpose :)

[BeyondEvil]

heh, ok.

I'll explore the behaviour in 3.x and compare.

[jeffwright13]

From my point of view, only the logs of the last (re)run matter.

I would think the exact opposite, myself. Why don't you care seeing logs from previous failures that force a rerun? Those test cases are the most flaky, and therefore of the most interest.

[orgads]

Right, but if it failed 3 times, and each run produces many logs, having the logs duplicated is cumbersome. I assume that if all runs failed, then the last run shouldn't be much different than the previous ones.

[jeffwright13]

We're running a plugin called pytest-rerunfailures. We might expect to see output from reruns :-)