Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

op: poll and poll_multi #255

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Commits on Feb 21, 2023

  1. BufRing: Provided Buffer Pool

    Adds new types: bufring::{BufRing, Builder}, bufgroup:{BufX, Bgid, Bid}
    
        (BufX was chosen as the name of the buffer type of this pool
        simply to make it short and easy to grep for.)
    
        bufgroup may be better named provbuf, or provided_buffers, down the
        road, especially if we intent to support the other, older, much less
        efficient provided buffer mechanism. But the liburing team
        specifically recommends the buf_ring implementation for ease of use
        and performance reasons so we may not want to incur the overhead of
        supporting both forms.
    
        bufring could become a trait down the road, where different BufRing
        implementations can be provided. But to avoid the extra decision
        making that comes with designing traits, and to keep the code
        relatively easy to follow, these are all concrate types.
    
    Adds register and unregister functionality to BufRing and to the
    tokio_uring driver.
    
    Adds experimental recv, recv_provbuf methods to TcpStream.
    
        recv_provbuf is a purposefully ugly name. It will be replaced by
        something ... maybe a recv builder sometimme soon.
    
        Whatever is chosen would then be copied for Udp, Unix and for all
        the other operations that optionally take a provided buffer pool id.
    
    Adds 'net' unit tests. All are admittedly simple ping/pong tests where
    only the clients' received lengths are checked, not the actual data.
    
    Adds a tests/common area.
    
    Adds a test case that uses two std::threads, where each thread runs its
    own tokio_uring runtime and its own buf_ring provided buffer pool.
    
        The two-thread case is made long, with many clients, sending many
        large messages to the server and getting them back, in order to see
        gross performance impacts when changing things. It takes 3s on my
        machine. Before going into mainline, the numbers would be changed so
        it took no more than the other unit tests, so about 10ms.
    
    Many TODOs left to cleanup. Primarily Safety rationalizations.
    
    The buffer allocation is made as a single allocation, along with the
    ring.
    
    The buffer group id, bgid, also sometimes called the provided buffer
    group id, is manually selected through the builder api. There is no
    mechanism to pick one automatically. That could be added later but is
    not really necessary for this feature to be useful.
    
    This first implementation is without traits and without public
    interfaces that would let a user create a different kind of buf_ring or
    a different kind of `provided buffers` pool.
    
    There's a question to the liburing team outstanding about how to
    interpret an unexpected cqe result of res=0 and flags=4.
    FrankReh committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    d2b3418 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    dbcbd8b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9ccadd8 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    300abc8 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a565029 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    0969da3 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d40d0a7 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    5978032 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    bd09dee View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    2029a20 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2023

  1. typo

    FrankReh committed Feb 22, 2023
    Configuration menu
    Copy the full SHA
    0f940c0 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2023

  1. Configuration menu
    Copy the full SHA
    9387c92 View commit details
    Browse the repository at this point in the history
  2. fix doc

    FrankReh committed Feb 24, 2023
    Configuration menu
    Copy the full SHA
    d885aa7 View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2023

  1. Configuration menu
    Copy the full SHA
    7a98190 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2023

  1. Configuration menu
    Copy the full SHA
    f6eafdb View commit details
    Browse the repository at this point in the history
  2. op: add recv_multi

    The first operation that supports streaming CQE results.
    
      Adds a Streamable trait, along the lines of the Completable trait.
        Comes with stream_next and stream_complete methods.
    
      Adds MultiCQEStream struct, along the lines of the MultiCQEFuture
      struct.
    
      Adds a submit_op_stream along the lines of submit_op.
    
      Adds poll_next_op along the lines of poll_op.
    
      The Lifecycle gets two additional methods: take_index, and data.
    
        take_index: returns the Lifecycle index and replaces it with
        usize::MAX to show the Lifecycle is no longer represented in the
        slab. This feature only used by the new poll_next_op.
    
        data: returns a reference to the Lifecycle's data, to be able to use
        it for the Streamable's stream_next calls which only require a
        reference. Ownership is still transferred in stream_complete.
    
    Adds the io/recv_multi operation.
    
    The tcp stream recv_multi is a function that
    bridges the private types with a public function.
    
    There is no cancel yet for the multishot command but the code can be
    written to break out of the loop. Also, when the connection is closed,
    the command should fail. It's not tested, but unregistering the buf_ring
    might cancel the command too - but maybe not.
    
    Unit tests:
    
      net unit tests and helper functions return io::Return
    
      add recv_multi cases
    
      add BufRingProps to net unit tests - to quiet empty buffer warnings
      which are intentional in some tests
    FrankReh committed Feb 26, 2023
    Configuration menu
    Copy the full SHA
    be7cdd4 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2023

  1. Configuration menu
    Copy the full SHA
    d26f34a View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2023

  1. op: add accept_multi for TcpListener

    And adds a unit test to tests/net.
    FrankReh committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    5139ae3 View commit details
    Browse the repository at this point in the history
  2. op: poll and poll_multi

    FrankReh committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    a89d404 View commit details
    Browse the repository at this point in the history