tests: Add a playbook and job to run sanity tests #122
Conversation
dmsimard
force-pushed
the
sanity-tests
branch
3 times, most recently
from
d2dae58
to
c96978a
Compare
I like this but it won't pick up 2.13 yet because it's a pre-release. I've gone ahead and committed the suggestion anyway (2.13 releases next week) but we can also figure something out (add |
This is meant to install every collection in a release's galaxy-requirements.yaml and then iterate over them to run sanity tests on each. Note that errors are not fatal yet (ignore_errors: true). Co-authored-by: David Moreau Simard <[email protected]> Co-authored-by: Felix Fontein <[email protected]>
| ansible_core_next: 2.13 | ||
| - name: latest Ansible 6 release | ||
| ansible_major: 6 | ||
| ansible_core: 2.13 |
There was a problem hiding this comment.
TODO: this won't work until 2.13 releases next week
| - name: Install ansible-core ${{ matrix.ansible_core }} | ||
| run: | | ||
| python3 -m pip install --upgrade pip | ||
| python3 -m pip install ansible-core>=${{ matrix.ansible_core }},${{ matrix.ansible_core_next }} |
There was a problem hiding this comment.
This way it should also install pre-releases of ansible-core, without installing pre-releases of ansible-core's dependencies:
| python3 -m pip install ansible-core>=${{ matrix.ansible_core }},${{ matrix.ansible_core_next }} | |
| python3 -m pip install ansible-core>=${{ matrix.ansible_core }}a1,${{ matrix.ansible_core_next }} |
There was a problem hiding this comment.
I'm curious and want to test it out of curiosity, let me get back to you on that.
| ANSIBLE_COLLECTIONS_PATH: "{{ collections_path }}" | ||
| command: ansible-galaxy collection install -r {{ galaxy_requirements }} | ||
| args: | ||
| creates: "{{ collections_path }}" |
There was a problem hiding this comment.
Need to add a retry here because it can (and will) fail like this:
________________________________________
< TASK [Install collections from galaxy] >
----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["ansible-galaxy", "collection", "install", "-r", "/root/src/ansible-build-data/6/galaxy-requirements.yaml"], "delta": "0:00:28.981410", "end": "2022-05-17 19:47:26.388533", "msg": "non-zero return code", "rc": 1, "start": "2022-05-17 19:46:57.407123", "stderr":
"ERROR! Error when getting the collection info for cloud.common from default (https://galaxy.ansible.com/api/) (HTTP Code: 502, Message: Bad Gateway Code: Unknown)", "stderr_lines": ["ERROR! Error when getting the collection info for cloud.common from default (https://galaxy.ansible.com/api/) (HTTP Code: 502, Message:
Bad Gateway Code: Unknown)"], "stdout": "Starting galaxy collection install process\nProcess install dependency map", "stdout_lines": ["Starting galaxy collection install process", "Process install dependency map"]}
There was a problem hiding this comment.
Can use the same approach as in antsibull: https://github.com/ansible-community/antsibull/blob/e8595b5d1dfc4c77dc99fb95927270fcbaa7a40a/roles/build-release/tasks/build.yaml#L36-L48
This is meant to install every collection in a release's
galaxy-requirements.yaml and then iterate over them to run sanity tests
on each.
Note that errors are not fatal yet (ignore_errors: true).