ZTS: Use QEMU for tests on Linux and FreeBSD #15838
Conversation
|
Most FreeBSD tests will get fixed via starting nfsd+samba I think. |
|
@mcmilk I see you currently have this marked as "Draft". When you think it's ready to be reviewed, please let us know and we can take a look. |
|
Seems ready, I included the FreeBSD |
mcmilk
force-pushed
the
qemu-machines
branch
3 times, most recently
from
03b7fec
to
79d3bb1
Compare
|
Note: I'm actively testing this PR in #16195. Right now I'm running down a bunch of test failures. |
I am back from holiday and will also help. I'll investigate the serial console thing first. |
mcmilk
force-pushed
the
qemu-machines
branch
2 times, most recently
from
c58d916
to
31fc59e
Compare
|
It's not final. The summary isn't ready and some debug things need to be removed. Can I leave the Ubuntu tests out?
I would like to add some SUSE distribution as well. |
mcmilk
force-pushed
the
qemu-machines
branch
3 times, most recently
from
3fd75b9
to
702642f
Compare
|
Just to make things easier (and not use so many runners), you can exclude the |
mcmilk
force-pushed
the
qemu-machines
branch
3 times, most recently
from
225a827
to
7000428
Compare
|
I think it's done now. We can remove the "Status: Work in Progress" badge.... @tonyhutter - What do you think? |
|
@mcmilk that's great news! I'll take a look once all the runners report back. |
I force pushed again and removed centos-stream-9 and some debugging things within the scripts. I have seen that you would like to split the tests into fractions like this: 1/3 2/3 ... do you want to add this later or is this just an idea? |
|
I have added FreeBSD 13.3 RELEASE and FreeBSD 14.1 RELEASE to the testings. |
Correct, right now it's just an idea. I think it might help with some timing-related failures like: I also vaguely remember buildbot giving me issues if I ran with instances that were less than 8GB RAM as well. That's why I'm curious if running 2 VMs with 8GB RAM might make many of this failures go away. I'm starting to get my variable-number-of-VMs code working with 2 VMs in my testing PR (tonyhutter#1), but I haven't gotten a full run working yet. Once I can get a full run with 2 VMs tested, I wanted to compare it's failures to the remaining failures in this PR. That will help us understand if the failures are timing/underpowered-VM related, or if we need to do some manual fixes to the tests. |
|
Oh no, I forgot the changed |
|
Almalinux 8+9, Debian and the FreeBSD 13+14 systems should go green now. |
mcmilk
force-pushed
the
qemu-machines
branch
2 times, most recently
from
d5e3d1d
to
0782fa5
Compare
|
It would be easier - and faster - if the github runners would have 16Gig more RAM. |
|
@mcmilk I think we might be missing some stderr output on the QEMU builders. For example, here's the same ZTS bug (#16439) on both builders: QEMU: https://github.com/openzfs/zfs/actions/runs/10388084683/job/28762944809 BUILDBOT: |
The test needs some adjusting within the timings. Signed-off-by: Tony Hutter <[email protected]> Co-authored-by: Tino Reichardt <[email protected]>
Sometimes the pool may start an auto scrub. Signed-off-by: Tino Reichardt <[email protected]> Signed-off-by: Tony Hutter <[email protected]>
This commit adds functional tests for these systems: - AlmaLinux 8, AlmaLinux 9, ArchLinux - CentOS Stream 9, Fedora 39, Fedora 40 - Debian 11, Debian 12 - FreeBSD 13, FreeBSD 14, FreeBSD 15 - Ubuntu 20.04, Ubuntu 22.04, Ubuntu 24.04 - enabled by default: - AlmaLinux 8, AlmaLinux 9 - Fedora 39, Fedora 40 - FreeBSD 13, FreeBSD 14, FreeBSD 15 Workflow for each operating system: - install qemu on the github runner - download current cloud image of operating system - start and init that image via cloud-init - install dependencies and poweroff system - start system and build openzfs and then poweroff again - clone build system and start 3 instances of it - the functional testings complete within times < 3h Signed-off-by: Tino Reichardt <[email protected]> Signed-off-by: Tony Hutter <[email protected]>
|
I fixed these things:
An older testrun with failing Fedora 39+40 is here: https://github.com/mcmilk/zfs/actions/runs/10414909636 TODO:
|
|
@mcmilk this will take care of the checkstyle issues: diff --git a/scripts/zfs-tests.sh b/scripts/zfs-tests.sh
index 957e674be..fde2e4acb 100755
--- a/scripts/zfs-tests.sh
+++ b/scripts/zfs-tests.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
# shellcheck disable=SC2154
#
# CDDL HEADER START
@@ -215,8 +215,8 @@ find_runfile() {
#
split_tags() {
# Get numerator and denominator
- NUM=$(echo $TAGS | cut -d/ -f1)
- DEN=$(echo $TAGS | cut -d/ -f2)
+ NUM=$(echo "$TAGS" | cut -d/ -f1)
+ DEN=$(echo "$TAGS" | cut -d/ -f2)
# At the point this is called, RUNFILES will contain a comma separated
# list of full paths to the runfiles, like:
#
@@ -242,9 +242,12 @@ split_tags() {
#
# "append,atime,bootfs,cachefile,checksum,cp_files,deadman,dos_attributes, ..."
- cat ${RUNFILES/,/ } | tr -d [],\' | awk '/tags = /{print $NF}' | sort | \
+ # Change the comma to a space for easy processing
+ _RUNFILES="$(echo """$RUNFILES""" | sed 's/,/ /g')"
+ # shellcheck disable=SC2002,SC2086
+ cat $_RUNFILES | tr -d "[],\'" | awk '/tags = /{print $NF}' | sort | \
uniq | grep -v functional | \
- awk -v num=$NUM -v den=$DEN '{ if(NR % den == (num - 1)) {printf "%s,",$0}}' | \
+ awk -v num="$NUM" -v den="$DEN" '{ if(NR % den == (num - 1)) {printf "%s,",$0}}' | \
sed -E 's/,$//'
}
@@ -568,7 +571,7 @@ RUNFILES=${R#,}
#
# "append,atime,bootfs,cachefile,checksum,cp_files,deadman,dos_attributes, ..."
#
-if echo $TAGS | grep -Eq '^[0-9]+/[0-9]+$' ; then
+if echo "$TAGS" | grep -Eq '^[0-9]+/[0-9]+$' ; then
TAGS=$(split_tags)
fi
|
|
I am testing zram disks again, it looks that they will speedup the whole thing a lot. The checkstyle fixups will get included, thank you. |
|
I will re-open another PR with this topic soon. Some more testings need to be done first - sry. |
|
@mcmilk I'm excited to see the new version! |
|
For those following - a newer version of the this PR got merged: #16537 |
Motivation and Context
We have the need for more tests on operating systems != Ubuntu.
Description
ZTS: Use QEMU for tests on Linux and FreeBSD...
This commit adds functional tests for these systems:
AlmaLinux 8, AlmaLinux 9, ArchLinux
CentOS Stream 9, Fedora 39, Fedora 40
Debian 11, Debian 12
FreeBSD 13, FreeBSD 14, FreeBSD 15
Ubuntu 20.04, Ubuntu 22.04, Ubuntu 24.04
enabled by default:
Workflow for each operating system:
Types of changes
Checklist:
Signed-off-by.