Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: use shared ArrayBuffer for nextv() #44

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Commits on Jun 19, 2022

  1. POC: use shared ArrayBuffer for nextv()

    This makes `db.iterator()` with buffer encoding as fast as an
    iterator with utf8 encoding. The approach is:
    
    1. In each `nextv()` call, create a `std::vector<char>` to hold the
       raw data of multiple entries
    2. Copy LevelDB slices directly into that vector with `memcpy()`
    3. Create an ArrayBuffer backed by the vector
    4. In JS, split it into Buffers, each backed by the same ArrayBuffer
       but using a different offset.
    
    Apart from this being an incomplete implementation (it makes utf8
    slower because the C++ side is buffer-only meaning JS has to
    transcode buffer to utf8), the approach has a downside: if userland
    code keeps a reference to just one of the Buffers, the entire
    ArrayBuffer is kept alive too. I.e. it costs memory.
    
    For now this PR is just a reference. The ideal solution (for this
    particular bottleneck) sits somewhere in between. For example, I
    might take just the ArrayBuffer concept, to replace use of
    `napi_create_buffer_copy()`.
    vweevers committed Jun 19, 2022
    Configuration menu
    Copy the full SHA
    57280fe View commit details
    Browse the repository at this point in the history