Draid and compression

[aheahe]

I have create 4 testpools.

zraid0
zraid1
zraid2
draid

I have set compression=zstd on all 4 pools.

I then copyed the same dataset to all 4 pools.

On the draid pool, the USED space is larger and the compressratio is smaller, which seems weird

NAME USED AVAIL REFER MOUNTPOINT
dr-test/prox190-rpool-root-pve1 9.71G 41.1G 9.71G /dr-test/prox190-rpool-root-pve1
zraid0/prox190-rpool-root-pve1 7.11G 30.7G 7.11G /zraid0/prox190-rpool-root-pve1
zraid1/prox190-rpool-root-pve1 7.11G 38.9G 7.11G /zraid1/prox190-rpool-root-pve1
zraid2/prox190-rpool-root-pve1 7.11G 38.9G 7.11G /zraid2/prox190-rpool-root-pve1

Compression is ZSTD on all 4 pools.

But the compressratio is diffrent

zfs get compressratio dr-test/prox190-rpool-root-pve1 zraid0/prox190-rpool-root-pve1 zraid1/prox190-rpool-root-pve1 zraid2/prox190-rpool-root-pve1
NAME PROPERTY VALUE SOURCE
dr-test/prox190-rpool-root-pve1 compressratio 1.24x -
zraid0/prox190-rpool-root-pve1 compressratio 1.37x -
zraid1/prox190-rpool-root-pve1 compressratio 1.37x -
zraid2/prox190-rpool-root-pve1 compressratio 1.37x -

` zpool status zraid0 zraid1 zraid2 dr-test
pool: dr-test
state: ONLINE
config:

NAME                  STATE     READ WRITE CKSUM
dr-test               ONLINE       0     0     0
  draid1:8d:12c:0s-0  ONLINE       0     0     0
    /tmp/draid-1      ONLINE       0     0     0
    /tmp/draid-2      ONLINE       0     0     0
    /tmp/draid-3      ONLINE       0     0     0
    /tmp/draid-4      ONLINE       0     0     0
    /tmp/draid-5      ONLINE       0     0     0
    /tmp/draid-6      ONLINE       0     0     0
    /tmp/draid-7      ONLINE       0     0     0
    /tmp/draid-8      ONLINE       0     0     0
    /tmp/draid-9      ONLINE       0     0     0
    /tmp/draid-10     ONLINE       0     0     0
    /tmp/draid-11     ONLINE       0     0     0
    /tmp/draid-12     ONLINE       0     0     0

errors: No known data errors

pool: zraid0
state: ONLINE
config:

NAME             STATE     READ WRITE CKSUM
zraid0           ONLINE       0     0     0
  /tmp/zraid0-1  ONLINE       0     0     0
  /tmp/zraid0-2  ONLINE       0     0     0

errors: No known data errors

pool: zraid1
state: ONLINE
config:

NAME             STATE     READ WRITE CKSUM
zraid1           ONLINE       0     0     0
  /tmp/zraid1-1  ONLINE       0     0     0
  /tmp/zraid1-2  ONLINE       0     0     0
  /tmp/zraid1-3  ONLINE       0     0     0
  /tmp/zraid1-4  ONLINE       0     0     0
  /tmp/zraid1-5  ONLINE       0     0     0

errors: No known data errors

pool: zraid2
state: ONLINE
config:

NAME             STATE     READ WRITE CKSUM
zraid2           ONLINE       0     0     0
  /tmp/zraid2-1  ONLINE       0     0     0
  /tmp/zraid2-2  ONLINE       0     0     0
  /tmp/zraid2-3  ONLINE       0     0     0
  /tmp/zraid2-4  ONLINE       0     0     0
  /tmp/zraid2-5  ONLINE       0     0     0

`

Is there a logical explanation to this - or is it a bug?

All pools created same option with "-o ashift=12"
Best Regards

[amotin]

DRAID by design can't allocate small blocks. I don't know that code path exactly, but guess it may propagate to compression layer. Increase of recordsize should in theory minimize the difference.

[aheahe]

It is not that.
I have tried to change the recordsize to both 1M and 512bytes. It makes no difference.

#zfs list
NAME      USED  AVAIL     REFER  MOUNTPOINT
dr-test  9.72G  41.1G      832K  /dr-test

zfs list zraid2
NAME     USED  AVAIL     REFER  MOUNTPOINT
zraid2  7.11G  38.9G       96K  /zraid2

Again, exactly the same data on a draid vs a zraid2 with zstd as compression.

[amotin]

It can't be the same between 1M and 512 bytes. The last should explode in size by order of magnitude, since neither RAIDZ nor especially DRAID can store blocks that small.

[amotin]

And BTW, recordsizes extend up to 16MB.

[aheahe]

Thank you for your help.

Regarding recordsize

zfs set recordsize=16M dr-test
cannot set property for 'dr-test': 'recordsize' must be power of 2 from 512B to 1M
root@wiz-store2:~# 

Regarding the 512 bytes size

zfs get recordsize dr-test
NAME     PROPERTY    VALUE    SOURCE
dr-test  recordsize  512      local
# zfs list dr-test
NAME      USED  AVAIL     REFER  MOUNTPOINT
dr-test  9.72G  41.1G      832K  /dr-test

ZFS is 2.1.5 is from Debian Backports on Debian 11

dr-test is backed by files for testing, maybe that makes the difference?

But i discovered this because i compared a raidz pool and a draid pool with the same data (backup) and it used a lot more space on the draid pool, which seemes weird to me. That were two servers with physical disks.

Best Regards

[amotin]

Just as a note, recordsize affects only files created after its was set. And It does not affect replication receive, received files keep their original record sizes, so to see the effect you must copy the data with cp or something alike.

[amotin]

ZFS is 2.1.5 is from Debian Backports on Debian 11

Ah, I see 2.1 branch still default zfs_max_recordsize parameter to 1MB, but it can be increased up to 16MB, same as it is on master.

[behlendorf]

@aheahe this behavior is expected for dRAID and described in the zpoolconcepts(7) man page.

              Unlike raidz, dRAID uses a fixed stripe width (padding as neces‐
              sary with zeros) to allow fully sequential resilvering.  This
              fixed stripe width significantly effects both usable capacity
              and IOPS.  For example, with the default D=8 and 4 KiB disk
              sectors the minimum allocation size is 32 KiB.  If using com‐
              pression, this relatively large allocation size can reduce the
              effective compression ratio.  When using ZFS volumes and dRAID,
              the default of the volblocksize property is increased to account
              for the allocation size.  If a dRAID pool will hold a signifi‐
              cant amount of small blocks, it is recommended to also add a
              mirrored special vdev to store those blocks.