Optimizing Vineyard for big payloads

[dv-rastogi]

Hi,
We are using Vineyard to write & read big payloads (~GB). Are there any configurations / tweaks one can use for maximizing performance there? For example, maybe one can optimize the underlying memory allocator for allocating much larger blocks thus reducing the allocator overhead?

[sighingnow]

Hi @watch24hrs-iiitd,

Do you mean single large blobs or many small blobs but the summarization of size can be large?

[dv-rastogi]

Hi,
I meant single large blobs.

[sighingnow]

Sorry for missing this message.

[dv-rastogi]

Hi,

Any suggestions for the above?

[sighingnow]

For single large blobs, the main cost of creating blobs is allocating physical pages (the first run) and memcpy.

• for the first issue, vineyard supports a command line flags -reserve_memory to pre-allocate physical pages from OS, for a fresh launched vineyardd, this flag can reduce the time of creating 8G blob from 3.6s to 2.8s.

• for the second issue, we haven't invested much to provide memcpy utilities yet, but that could be optimized by avoiding the memcpy by refactoring to first creating blob builders and filling you logic data into the blob to avoid the extra memcpy.

[sighingnow]

We have provided a client-side allocator as well to allow applications allocate memory directly on vineyard's shared memory region but that are still in very early stage, see also

• https://github.com/v6d-io/v6d/blob/main/modules/malloc/malloc_wrapper.h
• https://github.com/v6d-io/v6d/blob/main/modules/malloc/mimalloc_allocator.h

[dv-rastogi]

Thank you, will look into this!

[dv-rastogi]

Furthermore, can we optimize anything on the read end for large payloads?

[sighingnow]

Read blob from vineyard is zero-copy with mmap() so getting large blob from vineyard should yields no extra overhead.

[sighingnow]

If the get blob performance is a bottleneck in your case, you can share the case and we could take a look into the trouble.

[dv-rastogi]

Read isn't a bottleneck in our case, although we were just looking if anything specific could be done to boost it even further. Thank you for your help!