Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: add benchmark for Compact() (and possible performance improvements) #517

Open
mitake opened this issue May 29, 2023 · 1 comment
Open

Comments

@mitake
Copy link

mitake commented May 29, 2023

Currently bbolt doesn't have benchmarks for Compact() function (which will be used for etcd's defrag: etcd-io/etcd#15470).

Performance of compaction is important. It's one of the top contributors for 5xx of k8s API server. So I think it's valuable to add some benchmarks of the compaction logic for understanding its performance properties and possible improvements. There might be some interesting parameters and possible ideas which are valuable to check, like:

  • how does FillPercent affect write throughput and disk efficiency?
  • if the compaction process can skip fdatasync() during intermediate tx, how will this affect throughput?
  • can bulk-loading of B+tree improve throughput?

Of course corruption issues should be higher priority but I think it's valuable to have some benchmarks.

cc @ahrtr @serathius @ptabor @chaochn47 @cenkalti @geetasg

@github-actions github-actions bot added the stale label Apr 17, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2024
@ahrtr ahrtr reopened this May 9, 2024
@ahrtr ahrtr removed the stale label May 9, 2024
@github-actions github-actions bot added the stale label Aug 8, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 12, 2024
@ahrtr ahrtr reopened this Dec 12, 2024
@serathius
Copy link
Member

Interestingly I have seen Compact causing write performance degradation, my expectations is higher FillPercent causes writes require more page modification. cc @AwesomePatrol

@github-actions github-actions bot removed the stale label Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

4 participants