reorganize randomly across more things

[graingert]

would it be possible to get pytest-randomly to shuffle based on overlapping fixture scopes rather than just module, class, fn

[adamchainz]

Would be cool. I wonder whether we should support different "shuffle modes".

[jacobian91]

I also would like to see this, there is a pytest plug-in that does this (pytest-random-order) but I would prefer to keep using randomly. I do like the different modes, and the 'global' version is especially interesting to me right now since I've seen in some recent tests my mocked objects 'living longer' than their test and affecting other tests in separate files entirely. I rewrote those tests to no longer have issues, but the only way I was able to detect it was to run it multiple times and a certain order of test files would cause the issue.

[adamchainz]

pytest-random-order is made by friend @jbasko :)

I rewrote those tests to no longer have issues, but the only way I was able to detect it was to run it multiple times and a certain order of test files would cause the issue.

Great!