Limit reruns per scope (module, class)

[amezin]

I have a test suite where all tests are somewhat flaky. I. e. 1-2 tests out of ~4000 fail almost every time. So I'd like to account for exactly that - rerun up to 2 tests, then fail.

How could the API look like? pytest.mark.flaky on a class/module just marks all tests as flaky - and should likely continue working this way.