[WIP]Make cloud providers dynamic #15537
Draft
+120
−81
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
djyasin
changed the title
webknjaz
reviewed
Sep 19, 2024
|
Let's untangle this ball of yarn... PYTHONDONTWRITEBYTECODE=1 py.test -p no:cacheprovider --create-db --cov --cov-report=xml --junitxml=reports/junit.xml awx/main/tests/unit awx/main/tests/functional awx/conf/tests awx/sso/testsSo this is what's failing ^ And below is a traceback chain: Traceback (most recent call last):
File "/usr/lib64/python3.11/logging/config.py", line 400, in resolve
found = getattr(found, frag)
^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'awx.main' has no attribute 'utils'This one is not fully clear where it originates. Perhaps, it's pytest's collection stage. During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.11/logging/config.py", line 552, in configure
formatters[name] = self.configure_formatter(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/logging/config.py", line 664, in configure_formatter
result = self.configure_custom(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/logging/config.py", line 479, in configure_custom
c = self.resolve(c)
^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/logging/config.py", line 402, in resolve
self.importer(used)
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/utils/__init__.py", line 14, in <module>
from awx.main.utils.licensing import get_licenser # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/utils/licensing.py", line 38, in <module>
from awx.main.constants import SUBSCRIPTION_USAGE_MODEL_UNIQUE_HOSTS
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/constants.py", line 8, in <module>
from awx.main.models.inventory import InventorySourceOptions
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/awx_devel/awx/main/models/__init__.py", line 12, in <module>
from ansible_base.resource_registry.fields import AnsibleResourceField
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/ansible_base/resource_registry/fields.py", line 1, in <module>
from django.contrib.contenttypes.models import ContentType
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/assertion/rewrite.py", line 184, in exec_module
exec(co, module.__dict__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/contrib/contenttypes/models.py", line 139, in <module>
class ContentType(models.Model):
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/base.py", line 129, in __new__
app_config = apps.get_containing_app_config(module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/apps/registry.py", line 260, in get_containing_app_config
self.check_apps_ready()
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/apps/registry.py", line 138, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.This is an import happening before the django things got initialized ^ The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/bin/py.test", line 8, in <module>
sys.exit(console_main())
^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 201, in console_main
code = main()
^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 156, in main
config = _prepareconfig(args, plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 341, in _prepareconfig
config = pluginmanager.hook.pytest_cmdline_parse(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
config = yield
^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 1140, in pytest_cmdline_parse
self.parse(args)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 1494, in parse
self._preparse(args, addopts=addopts)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/config/__init__.py", line 1398, in _preparse
self.hook.pytest_load_initial_conftests(
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/warnings.py", line 151, in pytest_load_initial_conftests
return (yield)
^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/_pytest/capture.py", line 154, in pytest_load_initial_conftests
yield
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pytest_django/plugin.py", line 361, in pytest_load_initial_conftests
_setup_django(early_config)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/pytest_django/plugin.py", line 237, in _setup_django
django.setup()
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/utils/log.py", line 76, in configure_logging
logging_config_func(logging_settings)
File "/usr/lib64/python3.11/logging/config.py", line 823, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib64/python3.11/logging/config.py", line 555, in configure
raise ValueError('Unable to configure '
ValueError: Unable to configure formatter 'json'And this is the last one that cascaded from the previous two ^ I see I think that before this new import |
webknjaz
reviewed
Sep 20, 2024
webknjaz
reviewed
Sep 20, 2024
webknjaz
reviewed
Sep 20, 2024
webknjaz
reviewed
Sep 24, 2024
webknjaz
reviewed
Sep 24, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 25, 2024
webknjaz
reviewed
Sep 27, 2024
webknjaz
reviewed
Sep 27, 2024
webknjaz
reviewed
Sep 27, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
webknjaz
reviewed
Oct 1, 2024
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
…e_limits_editable_fields
…ditable_fields and got it passing
djyasin
force-pushed
the
make_cloud_providers_dynamic_28722
branch
from
22f1f70
to
2a5875e
Compare
|
webknjaz
reviewed
Oct 16, 2024
| @@ -2482,6 +2490,7 @@ def get_field_from_model_or_attrs(fd): | |||
| else: | |||
| redundant_scm_fields = list(filter(lambda x: attrs.get(x, None), ['source_project', 'source_path', 'scm_branch'])) | |||
| if redundant_scm_fields: | |||
| breakpoint() | |||
There was a problem hiding this comment.
FIXME
Suggested change
| breakpoint() |
webknjaz
reviewed
Oct 16, 2024
| @@ -2300,6 +2301,7 @@ class Meta: | |||
|
|
|||
| class InventorySourceOptionsSerializer(BaseSerializer): | |||
| credential = DeprecatedCredentialField(help_text=_('Cloud credential to use for inventory updates.')) | |||
| # source = serializers.ChoiceField(choices=[]) | |||
webknjaz
reviewed
Oct 16, 2024
| :returns: Dictionary of plugin cloud names plus source control. | ||
| :rtype: dict[str, str] | ||
| """ | ||
| # Get the list of plugin names |
There was a problem hiding this comment.
Let's drop the comments before merging. They shouldn't repeat the code. If code is unobvious, rename functions and variables to make it relay the same message as this comment.
webknjaz
reviewed
Oct 16, 2024
Comment on lines
+36
to
+40
| # Create a dictionary of plugin names | ||
| cloud_sources = {plugin: plugin for plugin in plugins} | ||
| cloud_sources['scm'] = 'Source Control Management' | ||
|
|
||
| return cloud_sources # Return the complete dictionary |
There was a problem hiding this comment.
Let's compose the dict in one go:
Suggested change
| # Create a dictionary of plugin names | |
| cloud_sources = {plugin: plugin for plugin in plugins} | |
| cloud_sources['scm'] = 'Source Control Management' | |
| return cloud_sources # Return the complete dictionary | |
| return dict(zip(plugins, plugins), scm='Source Control Management') |
webknjaz
reviewed
Oct 16, 2024
| return list(InventorySourceOptions.injectors.keys()) | ||
|
|
||
|
|
||
| def compute_cloud_inventory_sources() -> dict[str, str]: |
There was a problem hiding this comment.
Seeing that it's called in several places, let's cache this function's return value too:
Suggested change
| def compute_cloud_inventory_sources() -> dict[str, str]: | |
| @cache | |
| def compute_cloud_inventory_sources() -> dict[str, str]: |
webknjaz
reviewed
Oct 16, 2024
|
|
||
|
|
||
| @pytest.fixture | ||
| def fixture_compute_cloud_inventory_sources(): |
There was a problem hiding this comment.
No need to use “fixture” in fixture names. It's common to use the produced thing name, as that's what's injected into the tests. Additionally, fixtures should have docstrings for pytest --collect-only --fixtures.
Suggested change
| def fixture_compute_cloud_inventory_sources(): | |
| def computed_cloud_inventory_sources() -> dict[str, str]: | |
| """A mapping of cloud inventory source names to labels.""" |
webknjaz
reviewed
Oct 16, 2024
Comment on lines
+372
to
+375
| else: | ||
| raise ValueError("No inventories found in the queryset") | ||
| else: | ||
| raise ValueError("No available cloud providers found") |
There was a problem hiding this comment.
Let's reduce the indentation where possible:
Suggested change
| else: | |
| raise ValueError("No inventories found in the queryset") | |
| else: | |
| raise ValueError("No available cloud providers found") | |
| raise ValueError("No inventories found in the queryset") | |
| raise ValueError("No available cloud providers found") |
webknjaz
reviewed
Oct 16, 2024
| @@ -609,23 +609,9 @@ def test_get_constructed_inventory(self, constructed_inventory, admin_user, get) | |||
| r = get(url=reverse('api:constructed_inventory_detail', kwargs={'pk': constructed_inventory.pk}), user=admin_user, expect=200) | |||
| assert r.data['update_cache_timeout'] == 53 | |||
|
|
|||
| def test_patch_constructed_inventory(self, constructed_inventory, admin_user, patch): | |||
| def test_patch_constructed_inventory_generated_source_limits_editable_fields(self, constructed_inventory, admin_user, project, patch, inventory): | |||
There was a problem hiding this comment.
For the unused arguments, inject those fixtures differently:
Suggested change
| def test_patch_constructed_inventory_generated_source_limits_editable_fields(self, constructed_inventory, admin_user, project, patch, inventory): | |
| @pytest.mark.usefixtures('inventory', 'project') | |
| def test_patch_constructed_inventory_generated_source_limits_editable_fields(self, constructed_inventory, admin_user, patch) -> None: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
ISSUE TYPE
COMPONENT NAME
AWX VERSION
ADDITIONAL INFORMATION