Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pref(rocksdb): enable async io for iterators #286

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

shumkov
Copy link
Member

@shumkov shumkov commented Mar 26, 2024

Issue being fixed or feature implemented

Writes are not blocking in RocksDB but reads are blocking. There is an asynchronous IO provided for multi-gets and iterators. That should speed up query operations in Drive which are utilizing iterators. It should increase CPU consumption though.

More details:

What was done?

  • Enabled async io for RocksDB iterators
  • Benchmarks to prove that this option speeds up Drive's queries.

How Has This Been Tested?

With tests.

Breaking Changes

None

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

@fominok
Copy link
Collaborator

fominok commented Dec 27, 2024

As far as I understand RocksDB loads more data than possibly needed to reduce the number of disk reads, and those pre-fetches are blocking. If this change allows it to load subsequent data while we're processing current data then I see no harm in it, though PR title is a bit misleading since we introduce no "async io" to GroveDB, just some background processing out of our control.

@QuantumExplorer what do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants