Skip to content

Commit

Permalink
test: integration tests
Browse files Browse the repository at this point in the history
Let's follow-up on 0a0b0fa and introduce a second Packit job that runs
integration tests on the just built RPMs in Testing Farm [0].

The test organization and execution is handled by tmt [1]. To not
unnecessarily pollute the dbus-broker's git root, the tmt root is
initialized in the test/integration/ subdirectory (meaning that for the
tmt commands to work correctly you need to be somewhere below this
directory).

Since there are no pre-existing integration tests, add a simple one that
runs dfuzzer on the org.freedesktop.systemd1 D-Bus interface to
exercise dbus-broker a bit. A short README file is provided as well with
instructions on how to easily add a new test case, and with some links
where to find more information about tmt.

[0] https://packit.dev/docs/configuration/upstream/tests
[1] https://tmt.readthedocs.io/en/stable/overview.html
  • Loading branch information
mrc0mmand authored and teg committed Mar 12, 2024
1 parent 0a0b0fa commit db8f3ca
Show file tree
Hide file tree
Showing 6 changed files with 114 additions and 6 deletions.
24 changes: 18 additions & 6 deletions .packit.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
# vi:ts=2 sw=2 et:
# vi: set ts=2 sw=2 et:
#
# Docs: https://packit.dev/docs/

Expand Down Expand Up @@ -35,8 +35,20 @@ jobs:
- job: copr_build
trigger: pull_request
targets:
- fedora-all-aarch64
- fedora-all-i386
- fedora-all-ppc64le
- fedora-all-s390x
- fedora-all-x86_64
- fedora-all-aarch64
- fedora-all-i386
- fedora-all-ppc64le
- fedora-all-s390x
- fedora-all-x86_64

- job: tests
trigger: pull_request
fmf_path: test/integration/
tmt_plan: upstream_ci
targets:
# Limit the Testing Farm jobs to the latest stable and rawhide releases to conserve resources
# See: https://packit.dev/docs/configuration#aliases
- fedora-latest-stable-aarch64
- fedora-latest-stable-x86_64
- fedora-rawhide-aarch64
- fedora-rawhide-x86_64
1 change: 1 addition & 0 deletions test/integration/.fmf/version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1
54 changes: 54 additions & 0 deletions test/integration/README
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# dbus-broker integration test suite

dbus-broker's integration test suite uses tmt (Test Management Tool [0]) to organize and run tests. Since tmt
offers a _lot_ of features, this document pinpoints the most _interesting_ ones to get stuff up and running
quickly.

## How to contribute

Creating a new test case is pretty simple:

```
$ cd test/integration
$ tmt test create --template=shell test/name
Test directory '/home/.../dbus-broker/test/integration/test/name' created.
Test metadata '/home/.../dbus-broker/test/integration/test/name/main.fmf' created.
Test script '/home/.../dbus-broker/test/integration/test/name/test.sh' created.
```

The newly created `test.sh` will be the actual test case, and `main.fmf` contains the test metadata, including
test summary & description, test dependencies, runtime, and so on. See [1] for more details.

After tweaking the test metadata it's usually a good idea to run `tmt lint test/name` to make sure that the
configuration is still valid:

```
$ tmt lint test/name
/test/name
pass C000 fmf node passes schema validation
pass C001 summary key is set and is reasonably long
...
```

To check if the test itself works as expected you can use `tmt run`:

```
$ cd test/name
$ tmt run -vvv --all provision --how local tests --name .
...
total: 1 test passed
```

The `tmt run` command is _very_ customizable (as is the rest of `tmt`). In this particular example we tell it
to run all steps (`--all`) and override the `provision` and `tests` steps to run just one particular test on
the local machine. As in previous cases, check the `tmt` documentation [0] and examples [2] for more details.

## Links

[0] https://tmt.readthedocs.io/en/stable/overview.html

[1] https://tmt.readthedocs.io/en/stable/spec/tests.html

[2] https://tmt.readthedocs.io/en/stable/examples.html#run

<!-- vim: set syntax=markdown tw=110 : -->
7 changes: 7 additions & 0 deletions test/integration/fuzz/dfuzzer/main.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
summary: Run dfuzzer on a couple of D-Bus interfaces
test: ./test.sh
recommend:
- dfuzzer
- systemd
- util-linux
duration: 30m
25 changes: 25 additions & 0 deletions test/integration/fuzz/dfuzzer/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash
# vi: set sw=4 ts=4 et tw=110:

set -eux
set -o pipefail

TEST_USER="dfuzzer$SRANDOM"

at_exit() {
userdel -rf "$TEST_USER"
}

trap at_exit EXIT
useradd "$TEST_USER"

dbus-broker --version
systemctl status --no-pager dbus-broker.service

# Run dfuzzer on the PID 1's D-Bus interface. Drop privileges while doing so, since here we're interested in
# the actual message broking instead of breaking systemd.
#
# org.freedesktop.systemd1 was picked here because its interface is very rich when it comes to function
# signatures. Also, it's fuzzed in upstream by dfuzzer as well, which should make the test less prone to fails
# due to issues on systemd's side.
setpriv --reuid="$TEST_USER" --init-group -- dfuzzer -v --buffer-limit=10240 --bus org.freedesktop.systemd1
9 changes: 9 additions & 0 deletions test/integration/plans/upstream_ci.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# vi: set sw=2 ts=2 et ft=yaml tw=80:

# This plan discovers and executes all (enabled) integration tests

summary: Upstream integration test suite
discover:
how: fmf
execute:
how: tmt

0 comments on commit db8f3ca

Please sign in to comment.