Different results running tests via pytest or tox locally

[K-Meech]

At the moment, running tests with just pytest, results in errors:

ERROR tests/tests/test_brainmapper/test_analysis.py - DeprecationWarning: Implicit None on return values is deprecated and will rai...
ERROR tests/tests/test_brainreg/test_transform.py - DeprecationWarning: Implicit None on return values is deprecated and will rai...
!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!

while running via tox succeeds.

This is due to pytest using filterwarnings under tool.pytest.ini_options to turn any unlisted warnings into errors. This isn't used when running via tox.

Should filterwarnings also be used for tox runs?

[adamltyson]

As a default, I think pytest & tox should "match". @alessandrofelder?

[alessandrofelder]

pytest & tox should "match".

Agreed. And by default, I think we'd like to always have filter_warnings turn any unlisted warnings into errors.

Should filterwarnings also be used for tox runs?

So yes? (and that maybe means some refactoring/adapting to dependency updates?

[K-Meech]

I looked into this some more, and it appears the issue isn't with filterwarnings after all. I checked and the filter warnings settings are shared properly between pytest and tox.

Rather it's an issue with a difference in versions of importlib_metadata between my development environment, and the clean environment that tox creates. This seems to be due to the installation of napari in my dev environment, but not in the tox environment.

[alessandrofelder]

OK, does that mean we can close this as unrelated, @K-Meech ?

[K-Meech]

Yes I think so - I have a working dev environment now excluding napari. It does mean that any environment including napari + brainglobe_utils will likely throw these deprecation warnings though - so I don't know if that affects any modules that depend on both?

[alessandrofelder]

We defo have many of those environments.
I managed to reproduce locally - do you understand why there is a difference in importlib_metadata versions?

[K-Meech]

I think it's because a clean environment without napari uses importlib.metadata from the python standard library, while napari installation causes importlib_metadata to be used instead. This is a backport of importlib.metadata making it compatible with earlier versions of python.

Napari brings in version 7.0.1 of this module, which corresponds to version 3.13 of python. As we're using version 3.10 in brainglobe, this means it's effectively upgrading importlib.metadata to the latest version, which has an additional warning not included in 3.10.

[alessandrofelder]

Interesting - excellent detective work, thank you!

I wonder how much work it will be (and whether it's worth it) to move to importlib_metadata.
I'd suggest ignoring those warnings if we can (now we know why they happen) in tox and pytest.

(Side note: we aim to support more than just 3.10, and AFAIK we succeed in doing this apart from tensorflow + python 3.11 issues on Windows, which should be resolved by brainglobe/cellfinder#377)

[K-Meech]

Thanks @alessandrofelder - are you aiming to support python 3.12 then? If so, then it would probably be better to add a fix for this warning, as I think it was introduced in 3.12 (and so would affect both importlib.metadata and importlib_metadata)

[alessandrofelder]

you aiming to support python 3.12 then?

Yes, ideally from April, dependencies and our workload allowing 🙂
You're right, we should fix this. If it's a lot to fix, we can temporarily add the ignoring of the warning so tests don't fail while we continue working on other stuff simultaneously.