Plugins loaded via PYTEST_PLUGINS or pytest_plugins are not reported

[mtelka]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

When I load a plugin using the PYTEST_ADDOPTS environment variable then the plugin is properly reported in the list of plugins:

$ env - PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTEST_ADDOPTS='-p randomly' pytest --setup-plan
============================= test session starts ==============================
platform sunos5 -- Python 3.9.19, pytest-8.2.2, pluggy-1.5.0
Using --randomly-seed=2747766829
rootdir: /tmp/test
plugins: randomly-3.15.0
collected 0 items                                                              

============================ no tests ran in 0.03s =============================
$

When I try to achieve the same using the PYTEST_PLUGINS environment variable then the plugin is not listed in the list of plugins:

$ env - PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTEST_PLUGINS=pytest_randomly pytest --setup-plan
============================= test session starts ==============================
platform sunos5 -- Python 3.9.19, pytest-8.2.2, pluggy-1.5.0
Using --randomly-seed=3835907952
rootdir: /tmp/test
collected 0 items                                                              

============================ no tests ran in 0.03s =============================
$

but the plugin is apparently properly loaded (please note --randomly-seed).

It would be great to see plugins loaded via the PYTEST_PLUGINS environment variable reported too to avoid confusion.

EDIT: the same problem is seen when the pytest_plugins global variable is used to load plugins.

[mtelka]

I found that with the following patch:

--- /usr/lib/python3.9/vendor-packages/_pytest/config/__init__.py.orig
+++ /usr/lib/python3.9/vendor-packages/_pytest/config/__init__.py
@@ -835,7 +835,7 @@
     ) -> None:
         plugins = _get_plugin_specs_as_list(spec)
         for import_spec in plugins:
-            self.import_plugin(import_spec)
+            self.import_plugin(import_spec, consider_entry_points=True)

     def import_plugin(self, modname: str, consider_entry_points: bool = False) -> None:
         """Import a plugin with ``modname``.

the randomly plugin is properly listed in the list of plugins (but I had to use PYTEST_PLUGINS=randomly instead of PYTEST_PLUGINS=pytest_randomly):

$ env - PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTEST_PLUGINS=randomly pytest --setup-plan
============================= test session starts ==============================
platform sunos5 -- Python 3.9.19, pytest-8.2.2, pluggy-1.5.0
Using --randomly-seed=1218696459
rootdir: /tmp/test
plugins: randomly-3.15.0
collected 0 items                                                              

============================ no tests ran in 0.03s =============================
$

[mtelka]

I found that with the following patch:

--- /usr/lib/python3.9/vendor-packages/_pytest/config/__init__.py.orig
+++ /usr/lib/python3.9/vendor-packages/_pytest/config/__init__.py
@@ -835,7 +835,7 @@
     ) -> None:
         plugins = _get_plugin_specs_as_list(spec)
         for import_spec in plugins:
-            self.import_plugin(import_spec)
+            self.import_plugin(import_spec, consider_entry_points=True)

     def import_plugin(self, modname: str, consider_entry_points: bool = False) -> None:
         """Import a plugin with ``modname``.

the randomly plugin is properly listed in the list of plugins (but I had to use PYTEST_PLUGINS=randomly instead of PYTEST_PLUGINS=pytest_randomly):

$ env - PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTEST_PLUGINS=randomly pytest --setup-plan
============================= test session starts ==============================
platform sunos5 -- Python 3.9.19, pytest-8.2.2, pluggy-1.5.0
Using --randomly-seed=1218696459
rootdir: /tmp/test
plugins: randomly-3.15.0
collected 0 items                                                              

============================ no tests ran in 0.03s =============================
$

This was split out to #12624.

[mtelka]

The problem seems to be in the import_plugin() function.