Lock tests to a specific worker

[boxed]

We have some tests that use an external resource that can't be used in parallel for the purposes of the tests. I would like a way to set a bunch of tests to only run on one worker. I'm not sure if this is covered by the ideas in #18

I think a simple approach would be to have a pytest marker pytest.mark.xdist_first_worker. One can think of generalizing this to mark.xdist_worker_gw0 etc, but I think this is probably just ripe for abuse and then you'll have to validate that you're trying to lock it to an existing worker (so gw_8 will give an error if you only have 3 workers). This seems like it could solve a bunch of the issues in #18 with a much simpler code change and much easier to document.

I tried to hack this together myself by looking at scheduler/load.py but I got a bit lost...

[nicoddemus]

Hi @boxed,

Hmmm this is an interesting suggestion.

It might conflict with the proposal in #18 though, because it would create a conflict if an user marks tests with mark.xdist_first_worker that should execute in separate workers because of the fixtures they use.

Can you describe in more details why you want to specify tests in separate modules to run into a specific worker? We have currently --dist=loadscope and --dist=loadfile which might suite your needs already.

[boxed]

In our case the external resource is a single sign on service that we hard reset around the tests. It turned out we can switch to loadfile because right now they are in one file by mistake but that is likely to change.

I agree there might be a conflict with #18 but on the other hand that seems overly complex and no one is working on it :P

[boxed]

Btw, if this ticket is implemented, it should start by distributing the locked jobs so as to maximize parallelization. I'm looking at a run with --dist=loadfile right now that is super slow at the end because just one worker is running. This would be easy to fix with this ticket.

[nicoddemus]

Glad to know that loadfile is enough for your needs. Agree that we should send all jobs bound to a worker immediately.

[boxed]

Actually I spoke too soon. Loadfile has other problems so we'll have to just exclude some stuff from the xdist run totally :( I'll still be an improve but this feature would be much nicer.

[nicoddemus]

I'm thinking that we should implement the proposal in #19 (comment) to at least allow some meta-data to travel from worker to master; this would open the possibility of implementing more complex custom schedulers, such as the one you propose.

I'm talking about the pytest_xdist_collect_modify_meta hook and allow schedulers to access the dict meta-data.

[nicoddemus]

Seems like this can be accomplished by using the xdist_group mark and using loadgroup: you just need to be careful and group all the tests that require the authentication fixture in the same group.

Closing for now, we can reopen if needed.