-
Notifications
You must be signed in to change notification settings - Fork 84
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Let's follow-up on 0a0b0fa and introduce a second Packit job that runs integration tests on the just built RPMs in Testing Farm [0]. The test organization and execution is handled by tmt [1]. To not unnecessarily pollute the dbus-broker's git root, the tmt root is initialized in the test/integration/ subdirectory (meaning that for the tmt commands to work correctly you need to be somewhere below this directory). Since there are no pre-existing integration tests, add a simple one that runs dfuzzer on the org.freedesktop.systemd1 D-Bus interface to exercise dbus-broker a bit. A short README file is provided as well with instructions on how to easily add a new test case, and with some links where to find more information about tmt. [0] https://packit.dev/docs/configuration/upstream/tests [1] https://tmt.readthedocs.io/en/stable/overview.html
- Loading branch information
Showing
6 changed files
with
114 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
# dbus-broker integration test suite | ||
|
||
dbus-broker's integration test suite uses tmt (Test Management Tool [0]) to organize and run tests. Since tmt | ||
offers a _lot_ of features, this document pinpoints the most _interesting_ ones to get stuff up and running | ||
quickly. | ||
|
||
## How to contribute | ||
|
||
Creating a new test case is pretty simple: | ||
|
||
``` | ||
$ cd test/integration | ||
$ tmt test create --template=shell test/name | ||
Test directory '/home/.../dbus-broker/test/integration/test/name' created. | ||
Test metadata '/home/.../dbus-broker/test/integration/test/name/main.fmf' created. | ||
Test script '/home/.../dbus-broker/test/integration/test/name/test.sh' created. | ||
``` | ||
|
||
The newly created `test.sh` will be the actual test case, and `main.fmf` contains the test metadata, including | ||
test summary & description, test dependencies, runtime, and so on. See [1] for more details. | ||
|
||
After tweaking the test metadata it's usually a good idea to run `tmt lint test/name` to make sure that the | ||
configuration is still valid: | ||
|
||
``` | ||
$ tmt lint test/name | ||
/test/name | ||
pass C000 fmf node passes schema validation | ||
pass C001 summary key is set and is reasonably long | ||
... | ||
``` | ||
|
||
To check if the test itself works as expected you can use `tmt run`: | ||
|
||
``` | ||
$ cd test/name | ||
$ tmt run -vvv --all provision --how local tests --name . | ||
... | ||
total: 1 test passed | ||
``` | ||
|
||
The `tmt run` command is _very_ customizable (as is the rest of `tmt`). In this particular example we tell it | ||
to run all steps (`--all`) and override the `provision` and `tests` steps to run just one particular test on | ||
the local machine. As in previous cases, check the `tmt` documentation [0] and examples [2] for more details. | ||
|
||
## Links | ||
|
||
[0] https://tmt.readthedocs.io/en/stable/overview.html | ||
|
||
[1] https://tmt.readthedocs.io/en/stable/spec/tests.html | ||
|
||
[2] https://tmt.readthedocs.io/en/stable/examples.html#run | ||
|
||
<!-- vim: set syntax=markdown tw=110 : --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
summary: Run dfuzzer on a couple of D-Bus interfaces | ||
test: ./test.sh | ||
recommend: | ||
- dfuzzer | ||
- systemd | ||
- util-linux | ||
duration: 30m |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
#!/bin/bash | ||
# vi: set sw=4 ts=4 et tw=110: | ||
|
||
set -eux | ||
set -o pipefail | ||
|
||
TEST_USER="dfuzzer$SRANDOM" | ||
|
||
at_exit() { | ||
userdel -rf "$TEST_USER" | ||
} | ||
|
||
trap at_exit EXIT | ||
useradd "$TEST_USER" | ||
|
||
dbus-broker --version | ||
systemctl status --no-pager dbus-broker.service | ||
|
||
# Run dfuzzer on the PID 1's D-Bus interface. Drop privileges while doing so, since here we're interested in | ||
# the actual message broking instead of breaking systemd. | ||
# | ||
# org.freedesktop.systemd1 was picked here because its interface is very rich when it comes to function | ||
# signatures. Also, it's fuzzed in upstream by dfuzzer as well, which should make the test less prone to fails | ||
# due to issues on systemd's side. | ||
setpriv --reuid="$TEST_USER" --init-group -- dfuzzer -v --buffer-limit=10240 --bus org.freedesktop.systemd1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# vi: set sw=2 ts=2 et ft=yaml tw=80: | ||
|
||
# This plan discovers and executes all (enabled) integration tests | ||
|
||
summary: Upstream integration test suite | ||
discover: | ||
how: fmf | ||
execute: | ||
how: tmt |