Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZTS: Apply small changes for speeding up the tests #15811

Merged
merged 1 commit into from
Jan 26, 2024

Conversation

mcmilk
Copy link
Contributor

@mcmilk mcmilk commented Jan 23, 2024

Motivation and Context

The Github Action Runner got some new hardware metrics.
We should use the provided and empty disk which is pre-mounted at /mnt now.

Disk1: 89GiB -> rootfs + bootfs with ~80MB/s -> don't care
Disk2: 64GiB -> /mnt with 420MB/s -> new testing ssd

Description

This commit will mount the new disk to /var/tmp and provide hopefully some speedups within our testings.

How Has This Been Tested?

We will see, how much seepup we can get... when the ZTS is done with it ;-)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@andrewc12
Copy link
Contributor

Do you have any performance comparisons for this change?
From what I could see it seemed a little bit slower.

@mcmilk
Copy link
Contributor Author

mcmilk commented Jan 24, 2024

Since last week, the runner hardware improved a lot.
Only disk io heavy testings will see a huge performance increase, like the zloop testings.

This is the comparison for a freshly created runner:

[mcmilk@lydia ~]$ ssh [email protected] -p 29936
Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 6.2.0-1018-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Wed Jan 24 06:25:46 UTC 2024

  System load:  0.05126953125      Users logged in:          0
  Usage of /:   64.0% of 83.16GB   IPv4 address for docker0: 172.17.0.1
  Memory usage: 25%                IPv4 address for eth0:    10.1.0.99
  Swap usage:   0%                 IPv4 address for virbr0:  192.168.122.1
  Processes:    154

Expanded Security Maintenance for Applications is not enabled.

33 updates can be applied immediately.
21 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

19 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esm

Last login: Wed Jan 24 06:25:47 2024 from 127.0.0.1
runner@fv-az1240-812:~$ sudo bash

root@fv-az1240-812:/home/runner# hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads: 288 MB in  3.31 seconds =  87.06 MB/sec

root@fv-az1240-812:/home/runner# hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 1278 MB in  3.01 seconds = 424.40 MB/sec

root@fv-az1240-812:/home/runner# mount | grep sd[ab]
/dev/sda1 on / type ext4 (rw,relatime,discard,errors=remount-ro)
/dev/sda15 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
/dev/sdb1 on /mnt type ext4 (rw,relatime)


# -> we reformat with ext4 + move it to /var/tmp
DEV="/dev/disk/azure/resource-part1"
sudo umount /mnt
sudo mkfs.ext4 -F $DEV
Copy link
Member

@gmelikov gmelikov Jan 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we already format and remount ext4 - maybe:

  • mount with noatime,barrier=0 ?
  • disable journal?
  • maybe compilation will be faster too from this ssd? If it's so - we may move code onto it
  • other speedups I've missed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mount with noatime,barrier=0 ?
disable journal?

These two are done .... thanks for pointing out.

maybe compilation will be faster too from this ssd? If it's so - we may move code onto it

Compiling is done once on startup in ~10..20 minutes. This should be okay.

The Github Action Runner got some new hardware metrics.  We should use
the provided and empty disk which is pre-mounted at /mnt now.

Disk1: 89GiB -> rootfs + bootfs with ~80MB/s -> don't care
Disk2: 64GiB -> /mnt with 420MB/s -> new testing ssd

This commit will mount the new disk to /var/tmp and provide hopefully
some speedups within our testings.

Signed-off-by: Tino Reichardt <[email protected]>
@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Jan 26, 2024
@behlendorf behlendorf merged commit dac0bae into openzfs:master Jan 26, 2024
21 of 25 checks passed
behlendorf pushed a commit to behlendorf/zfs that referenced this pull request Jan 29, 2024
The Github Action Runner got some new hardware metrics.  We should use
the provided and empty disk which is pre-mounted at /mnt now.

Disk1: 89GiB -> rootfs + bootfs with ~80MB/s -> don't care
Disk2: 64GiB -> /mnt with 420MB/s -> new testing ssd

This commit will mount the new disk to /var/tmp and provide hopefully
some speedups within our testings.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Andrew Innes <[email protected]>
Signed-off-by: Tino Reichardt <[email protected]>
Closes openzfs#15811
behlendorf pushed a commit that referenced this pull request Jan 29, 2024
The Github Action Runner got some new hardware metrics.  We should use
the provided and empty disk which is pre-mounted at /mnt now.

Disk1: 89GiB -> rootfs + bootfs with ~80MB/s -> don't care
Disk2: 64GiB -> /mnt with 420MB/s -> new testing ssd

This commit will mount the new disk to /var/tmp and provide hopefully
some speedups within our testings.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Andrew Innes <[email protected]>
Signed-off-by: Tino Reichardt <[email protected]>
Closes #15811
@mcmilk mcmilk deleted the zts-speedup branch January 30, 2024 18:27
@pcd1193182
Copy link
Contributor

I've noticed sporadic test failures in the code added in this PR; I'm not sure if it's a bug that's always been there, or if something has changed with the github workers, but a nontrivial number of test runs are failing as a result.

@mcmilk
Copy link
Contributor Author

mcmilk commented Mar 8, 2024

I noticed these sporadic test failures also.

I am in doubt if the other disk space is the reason for that - but we could test this, by reverting this PR maybe...
@behlendorf @gmelikov - should we try a revert here?

lundman pushed a commit to openzfsonwindows/openzfs that referenced this pull request Mar 13, 2024
The Github Action Runner got some new hardware metrics.  We should use
the provided and empty disk which is pre-mounted at /mnt now.

Disk1: 89GiB -> rootfs + bootfs with ~80MB/s -> don't care
Disk2: 64GiB -> /mnt with 420MB/s -> new testing ssd

This commit will mount the new disk to /var/tmp and provide hopefully
some speedups within our testings.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Andrew Innes <[email protected]>
Signed-off-by: Tino Reichardt <[email protected]>
Closes openzfs#15811
lundman pushed a commit to openzfsonwindows/openzfs that referenced this pull request Mar 13, 2024
The Github Action Runner got some new hardware metrics.  We should use
the provided and empty disk which is pre-mounted at /mnt now.

Disk1: 89GiB -> rootfs + bootfs with ~80MB/s -> don't care
Disk2: 64GiB -> /mnt with 420MB/s -> new testing ssd

This commit will mount the new disk to /var/tmp and provide hopefully
some speedups within our testings.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Andrew Innes <[email protected]>
Signed-off-by: Tino Reichardt <[email protected]>
Closes openzfs#15811
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants