Buffer pool tracking issue

[Berrysoft]

Prior Art

• add Buffer pool support #276 AsyncReadBufferPool::read_buffer_pool, AsyncReadAtBufferPool::read_at_buffer_pool, with BufferPool and Buffer associated types
• API of tokio-uring (read_fixed / write_fixed of TcpStream)

In Progress

• feat(io): buffer pool IO traits #356 a modified version of add Buffer pool support #276:
  • AsyncReadManaged::read_managed
  • AsyncReadManagedAt::read_managed_at
• feat(driver): buffer pool #358 (Implementation)

Any other advice or design?

[Berrysoft]

API of tokio-uring: https://docs.rs/tokio-uring/latest/tokio_uring/net/struct.TcpStream.html#method.read_fixed

The user get the buffer from the registry directly and pass the ownership to the read method.

[Berrysoft]

Also note that there's write_fixed in tokio-uring

[Berrysoft]

Note about io-uring internals:

• tokio-uring uses register_buffers and ReadFixed op
• add Buffer pool support #276 uses register_buf_ring and Read op

Buffer ring is a newer feature. It lets the kernel to choose an available buffer, which means the users cannot choose the buffer themselves, and should return the buffer after consuming it.

[Berrysoft]

The returned buffer might be better if it's owned:

pub trait AsyncReadManaged {
    type BufferPool;
    type Buffer: Deref<Target = [u8]>;

    async fn read_managed(
        &mut self,
        buffer_pool: &Self::BufferPool,
        len: usize,
    ) -> IoResult<Self::Buffer>;
}

The inner buffer pool might be Rc.

[George-Miao]

@Berrysoft Can you provide some examples? I'm not entirely sure how this is supposed to be used.

[Berrysoft]

The usage is like this:

let pool = BufferPool::new(/* count: */16, /* buffer len: */1024).unwrap();
let mut stream = TcpStream::connect(addr).await.unwrap();
// Use len = 0 to make the driver select one.
let buffer = stream.read_managed(&pool, 0).await.unwrap();
// Use buffer, and drop it to return it to the pool.

[George-Miao]

@Berrysoft So every IO depends on some specific type of BufferPool, is that right? If users want abstraction, they need to do it themselves.

[Berrysoft]

I think so. The buffer count & buffer len of the buffer pool is controled by the user, and they can even create many pools if they want. compio-runtime could provide a specific pool type, and TcpStream will only impl AsyncReadManaged based on compio_runtime::BufferPool.

[George-Miao]

What about this:

trait BufferPool {
    type Buffer;
}

trait AsyncReadManaged {
    type Pool: BufferPool + ?Sized;
    
    async fn read_managed(
        &mut self,
        buffer_pool: &Self::Pool,
        len: usize,
    ) -> IoResult<<Self::Pool as BufferPool>::Buffer>;
}

It's still flexibile (i.e., no other constraint on buffer pool, hell it can even be !Sized)

[Berrysoft]

What about this:

I have thought about this design, but didn't choose it because, there might be a case that the implementer of AsyncReadManaged wants to use a different Buffer type other than the one returned by BufferPool. If type Buffer is an assiciated type of trait BufferPool, the implementer need to author a new pool wrapper type to achieve this goal, which is more complex.