Skip to content

Commit

Permalink
ci: Add tags to TF workflow, allow more [citest bad] formats
Browse files Browse the repository at this point in the history 
* Add "BusinessUnit": "system_roles" environment setting to tag our jobs in Testing farm
* Add tmt_plan_filter to run additional workflows besides general
* Allow more [citest bad] comment formats
* Get memory and supported platforms info from the PR ref
* Move LINUXSYSTEMROLES_USER to vars and use it everywhere in tft.yml
* Remove extra GITHUB_ORG definition

Signed-off-by: Sergei Petrosian <[email protected]>
  • Loading branch information
@spetrosi
spetrosi committed Aug 28, 2024
1 parent 15dda14 commit e5c6920
Show file tree
Hide file tree
Showing 3 changed files with 131 additions and 9 deletions.
23 changes: 15 additions & 8 deletions .github/workflows/tft.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ jobs:
PR_NO: ${{ github.event.issue.number }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Checkout PR
uses: actions/checkout@v4
with:
ref: ${{ steps.head_sha.outputs.head_sha }}

- name: Get memory
id: memory
run: |
Expand Down Expand Up @@ -113,17 +118,17 @@ jobs:
ARTIFACTS_DIR_NAME: "tf_${{ github.event.repository.name }}-${{ github.event.issue.number }}_\
${{ matrix.platform }}-${{ matrix.ansible_version }}_\
${{ needs.prepare_vars.outputs.datetime }}/artifacts"
ARTIFACT_TARGET_DIR: /srv/pub/alt/linuxsystemroles/logs
ARTIFACT_TARGET_DIR: /srv/pub/alt/${{ vars.LINUXSYSTEMROLES_USER }}/logs
steps:
- name: Set variables with DATETIME and artifact location
id: set_vars
run: |
printf -v DATETIME '%(%Y%m%d-%H%M%S)T' -1
ARTIFACTS_DIR_NAME="tf_${{ github.event.repository.name }}-${{ github.event.issue.number }}_\
${{ matrix.platform }}-${{ matrix.ansible_version }}_$DATETIME/artifacts"
ARTIFACTS_TARGET_DIR=/srv/pub/alt/linuxsystemroles/logs
ARTIFACTS_TARGET_DIR=/srv/pub/alt/${{ vars.LINUXSYSTEMROLES_USER }}/logs
ARTIFACTS_DIR=$ARTIFACTS_TARGET_DIR/$ARTIFACTS_DIR_NAME
ARTIFACTS_URL=https://dl.fedoraproject.org/pub/alt/linuxsystemroles/logs/$ARTIFACTS_DIR_NAME
ARTIFACTS_URL=https://dl.fedoraproject.org/pub/alt/${{ vars.LINUXSYSTEMROLES_USER }}/logs/$ARTIFACTS_DIR_NAME
echo "DATETIME=$DATETIME" >> $GITHUB_OUTPUT
echo "ARTIFACTS_DIR=$ARTIFACTS_DIR" >> $GITHUB_OUTPUT
echo "ARTIFACTS_URL=$ARTIFACTS_URL" >> $GITHUB_OUTPUT
Expand Down Expand Up @@ -155,17 +160,18 @@ jobs:
git_url: https://github.com/linux-system-roles/tft-tests
git_ref: main
pipeline_settings: '{ "type": "tmt-multihost" }'
environment_settings: '{ "provisioning": { "tags": { "BusinessUnit": "system_roles" } } }'
# Keeping ARTIFACTS_URL at the bottom makes the link in logs clickable
variables: "ANSIBLE_VER=${{ matrix.ansible_version }};\
REPO_NAME=${{ github.event.repository.name }};\
GITHUB_ORG=${{ github.repository_owner }};\
GITHUB_ORG=linux-system-roles;\
PR_NUM=${{ github.event.issue.number }};\
ARTIFACTS_DIR=${{ steps.set_vars.outputs.ARTIFACTS_DIR }};\
ARTIFACTS_URL=${{ steps.set_vars.outputs.ARTIFACTS_URL }};\
TEST_LOCAL_CHANGES=false"
TEST_LOCAL_CHANGES=false;\
LINUXSYSTEMROLES_USER=${{ vars.LINUXSYSTEMROLES_USER }};\
ARTIFACTS_URL=${{ steps.set_vars.outputs.ARTIFACTS_URL }}"
# Note that LINUXSYSTEMROLES_SSH_KEY must be single-line, TF doesn't read multi-line variables fine.
secrets: "LINUXSYSTEMROLES_USER=${{ secrets.LINUXSYSTEMROLES_USER }};\
LINUXSYSTEMROLES_DOMAIN=${{ secrets.LINUXSYSTEMROLES_DOMAIN }};\
secrets: "LINUXSYSTEMROLES_DOMAIN=${{ secrets.LINUXSYSTEMROLES_DOMAIN }};\
LINUXSYSTEMROLES_SSH_KEY=${{ secrets.LINUXSYSTEMROLES_SSH_KEY }}"
compose: ${{ matrix.platform }}
# There are two blockers for using public ranch:
Expand All @@ -175,6 +181,7 @@ jobs:
api_key: ${{ secrets.TF_API_KEY_RH }}
update_pull_request_status: false
tmt_hardware: '{ "memory": ">= ${{ needs.prepare_vars.outputs.memory }} MB" }'
tmt_plan_filter: "tag:general,kernel_settings"

- name: Set final commit status
uses: myrotvorets/set-commit-status-action@master
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tft_citest_bad.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
citest_bad_rerun:
if: |
github.event.issue.pull_request
&& github.event.comment.body == '[citest_bad]'
&& contains(fromJson('["[citest_bad]", "[citest-bad]", "[citest bad]"]'), github.event.comment.body)
&& contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR"]'), github.event.comment.author_association)
permissions:
actions: write # for re-running failed jobs: https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2022-11-28#re-run-a-job-from-a-workflow-run
Expand Down
115 changes: 115 additions & 0 deletions contributing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,118 @@ are likely to be suitable for new contributors!

**Code** is managed on [Github](https://github.com/linux-system-roles/kernel_settings), using
[Pull Requests](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests).

## Python Code

The Python code needs to be **compatible with the Python versions supported by
the role platform**.

For example, see [meta](https://github.com/linux-system-roles/kernel_settings/blob/main/meta/main.yml)
for the platforms supported by the role.

If the role provides Ansible modules (code in `library/` or `module_utils/`) -
these run on the *managed* node, and typically[1] use the default system python:

* EL6 - python 2.6
* EL7 - python 2.7 or python 3.6 in some cases
* EL8 - python 3.6
* EL9 - python 3.9

If the role provides some other sort of Ansible plugin such as a filter, test,
etc. - these run on the *control* node and typically use whatever version of
python that Ansible uses, which in many cases is *not* the system python, and
may be a modularity release such as python311.

In general, it is a good idea to ensure the role python code works on all
versions of python supported by `tox-lsr` from py36 on, and on py27 if the role
supports EL7, and on py26 if the role supports EL6.[1]

[1] Advanced users may set
[ansible_python_interpreter](https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html#term-ansible_python_interpreter)
to use a non-system python on the managed node, so it is a good idea to ensure
your code has broad python version compatibility, and do not assume your code
will only ever be run with the default system python.

## Testing kernel_settings modules

It is recommended to use `tox` to set up your `virtualenv` for
development/testing purposes:

```bash
dnf/yum install python-tox
tox -e py38
```

You can also use the virtualenv created by `tox` just like any
other virtualenv created by `python-virtualenv`:

```bash
. .tox/env-py38/bin/activate
python
>>> import package.that.only.exists.in.venv
```

The unit tests and other tests are run by default when you use `tox` by itself
or `tox -e py38` for a specific python versioned environment. Note that other
operating system packages may be required to be installed in order for `tox`
to use `pip` to install python dependencies e.g. for python packages which
have native components.

I would also strongly encourage you to use an IDE for development. For example,
Visual Studio code python extension auto-discovers tests and allows you to
run and debug unit tests. However, you may need to create a `.env` file like
this, in order for code navigation, auto-completion, and test discovery to
work correctly:

```bash
PYTHONPATH=/full/path/to/tuned:/full/path/to/linux-system-roles/kernel_settings/library
```

### Testing the module

[Ansible Module Development Guide](https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html)

Using a tox python virtualenv from the `kernel_settings` directory:

```bash
. .tox/env-py38/bin/activate
TESTING=true [TEST_PROFILE=kernel_settings] python \
library/kernel_settings.py args.json
```

looks for test profiles under `tests/tuned/etc/tuned`

to run the code in the debugger:

```bash
TESTING=true [TEST_PROFILE=kernel_settings] python -mpdb \
library/kernel_settings.py args.json
```

Where `args.json` looks like this:

```json
{
"ANSIBLE_MODULE_ARGS": {
"name": "kernel_settings",
"sysctl": [
{"name": "fs.inotify.max_user_watches", "value": 524288},
{"name": "kernel.threads-max", "value": 30001}
],
"sysfs": [
{"name": "/sys/kernel/kexec_crash_size", "value": 337641472}
],
"bootloader": [
{"name": "cmdline", "value": [
{"name": "mitigations", "value": "on"},
{"name": "another"}
]
}
],
"selinux": [
{"name": "avc_cache_threshold", "value": 512}
],
"purge": false
}
}
```

0 comments on commit e5c6920

Please sign in to comment.