groot/internal/rcompress: reuse buffers when possible #1023
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #712, but the current implementation is a bit delicate (from pre-generics
sync
use), so I'm not entirely sure it's worth it for the marginal performance improvements.This reuses buffers for
lz4
,zlib
andzstd
. The other compression algs did not appear to expose an API allowing for buffer reuse. All of this should be threadsafe, but hit hasn't been tested very thoroughly. Nothing is flagged if I build groot-bench with-race
and generate files or run benchmarks, for whatever that's worth.Any performance gains would come from reducing allocations, seen in e.g.
go tool pprof -alloc-space
on a memprofile, along with a reduction in gc frequency from the reduced allocation rate. There's very little change to overall throughput or memory use (small improvements in some cases):