How to tune performance

[unkownerror1]

Hey everyone，
I am currently testing zfs performance using vdbench.When I changed the zfs attribute from sync=always to sync=disabled, the average iops increased from 5.5w to 7.7w.
Here’s what I’ve tried so far:

zfs_txg_timeout：15
zfs_dirty_data_max：68719476736
zfs_dirty_data_sync_percent：50

it was able to increase to 16w, but it fluctuated greatly.

Why does the performance fluctuate so much, or can someone tell me how to tune it.

kernel:4.19.90-2112.8.0.0131.oe1.x86_64
zfs:2.1.5
mem:89G

Thanks in advance for any help!

[amotin]

Performance optimization is a difficult thing with endless number of factors. Please do not look for a magic tuning, since those should already be applied as defaults. Instead focus on factors that are specific to your configuration.

For example, you use recordsize of 128KB, while your test seems to use 8KB writes. If the writes are random, it may cause huge write inflation and/or read-modify-write overheads. Please make sure you are testing something that is relevant for your workload, and that your pool/dataset are configured reasonably for both.

It is not surprising that changing of sync from always to disabled improves write performance. But the question it puts is whether you really need sync=always in the first part. Not every workload needs it. Then, I see you have ZLOG device, but set logbias=throughput, which makes most of data to be written onto the main pool instead of SLOG, which may not be very beneficial.

You also have significantly boosted transaction time and the dirty data limit. Those can possibly contribute to the speed fluctuation period, making it more noticeable. I've seen myself when some of disks complete writes within TXG sooner than others, which may result in write speed fluctuation, but I haven't got to looking on it closer yet.

And finaly, you are running very old kernel and pretty old ZFS, which means you may not benefit from any later performance optimizations.