Skip to content

Commit

Permalink
Update design documentation
Browse files Browse the repository at this point in the history 
Signed-off-by: Andrew Carbonetto <[email protected]>
  • Loading branch information
@acarbonetto
acarbonetto committed Nov 21, 2023
1 parent bb282ed commit 4c30c24
Showing 1 changed file with 154 additions and 118 deletions.
272 changes: 154 additions & 118 deletions docs/design-raw-ffi.md
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,58 @@
# Babushka Socket Listener
# Babushka Core Wrappers

## Unix Domain Socket Manager
## Summary

### Sequence Diagram
The Babushka client enables Redis users connect to Redis using a variety of languages and supported commands. The client
uses a performant core to establish and manage connections, and a thin wrapper layer written in various languages, to
communicate

**Summary**: The Babushka "UDS" solution uses a socket manager to redis-client worker threads, and UDS to manage the communication
between the wrapper and redis-client threads. This works well because we allow the socket to manage the communication. This
results in simple/fast communication. But the worry is that the unix sockets can become a bottleneck for data-intensive communication;
ie read/write buffer operations become the bottleneck.
The following document discusses two primary communication protocol architectures for wrapping the babushka clients, and
how Java-Babushka and Go-Babushka use each of those protocols differently by taking advantage of language-specific attributes.

_Observation_: We noticed that we are creating a fixed number of Babushka/Redis client connections based on the number of CPUs available.
It would be better to configure this thread count, or default it to the number of CPUs available.
### References

* Babushka - communication protocol architecture: describes high-level approaches to communication protocols.

# Unix Domain Socket Manager Connector

## High-Level Design

**Summary**: The Babushka "UDS" solution uses a socket listener to manage rust-to-wrapper worker threads, and unix domain sockets
to deliver package between the wrapper and redis-client threads. This works well because we allow the unix sockets to pass messages
through the OS and is very performant. This results in simple/fast communication. But the concern is that the unix sockets can
become a bottleneck for data-intensive communication; ie read/write buffer operations.

```mermaid
stateDiagram-v2
direction LR
JavaWrapper: Java Wrapper
UnixDomainSocket: Unix Domain Socket
RustCore: Rust-Core
[*] --> JavaWrapper: User
JavaWrapper --> UnixDomainSocket
UnixDomainSocket --> JavaWrapper
RustCore --> UnixDomainSocket
UnixDomainSocket --> RustCore
RustCore --> Redis
```

## Decision to use UDS Sockets for a Java-Babushka Wrapper

The decision to use Unix Domain Sockets (UDS) to manage the Java-wrapper to Babushka Redis-client communication was two-fold, as opposed to a raw-FFI solution:
1. Java contains an efficient socket protocol library (netty.io) that provides a highly configurable environment to manage sockets.
2. Java objects serialization/de-serialization is an expensive operation, and a performing multiple io operations between raw-ffi calls would be inefficient.

### Decision Log

| Protocol | Details | Pros | Cons |
|----------------------------------------------|---------------------------------------------------------|-----------------------------|----------------------------------------------------|
| Unix Domain Sockets (jni/netty) | JNI to submit commands; netty.io for message passing | netty.io standard lib; | complex configuration; limited by socket interface |
| Raw-FFI (JNA, uniffi-rs, j4rs, interoptopus) | JNA to submit commands; protobuf for message processing | reusable in other languages | Slow performance and uses JNI under the hood; |
| Panama/jextract | Performance similar to a raw-ffi using JNI | | lacks early Java support (JDK 18+); prototype |

### Sequence Diagram

```mermaid
sequenceDiagram
Expand All @@ -24,160 +66,154 @@ participant Socket as Unix Domain Socket
participant Client as Redis
activate Wrapper
activate Socket
activate Client
Wrapper ->>+ ffi: connect_to_redis
ffi ->>+ manager: start_socket_listener(init_callback)
Wrapper -)+ ffi: connect_to_redis
ffi -)+ manager: start_socket_listener(init_callback)
manager ->> worker: Create Tokio::Runtime (count: CPUs)
activate worker
worker ->> SocketListener: listen_on_socket(init_callback)
SocketListener ->> SocketListener: loop: listen_on_client_stream
activate SocketListener
manager -->> ffi: socket_path
ffi -->>- Wrapper: socket_path
SocketListener -->> Client: BabushkaClient::new
SocketListener -->> Socket: UnixStreamListener::new
Wrapper ->> Socket: write buffer (socket_path)
Socket -->> Wrapper:
SocketListener ->> SocketListener: handle_request
SocketListener ->> Socket: read_values_loop(client_listener, client)
Socket -->> SocketListener:
SocketListener ->> Client: send(request)
Client -->> SocketListener: ClientUsageResult
SocketListener ->> Socket: write_result
Socket -->> SocketListener:
Wrapper ->> Socket: read buffer (socket_path)
SocketListener -->> Socket: UnixStreamListener::new
activate Socket
SocketListener -->> Client: BabushkaClient::new
Wrapper ->> Socket: connect
Socket -->> Wrapper:
loop single_request
Wrapper ->> Socket: netty.writeandflush (protobuf.redis_request)
Socket -->> Wrapper:
Wrapper ->> Wrapper: Result
Wrapper ->> ffi: close_connection
ffi ->> manager: close_connection
manager ->>- worker: close
worker ->>SocketListener: close
deactivate SocketListener
deactivate worker
deactivate Wrapper
deactivate Client
deactivate Socket
Wrapper ->> Wrapper: wait
SocketListener ->> SocketListener: handle_request
SocketListener ->> Socket: read_values_loop(client_listener, client)
Socket -->> SocketListener:
SocketListener ->> Client: send(request)
Client -->> SocketListener: ClientUsageResult
SocketListener ->> Socket: write_result
Socket -->> SocketListener:
Wrapper ->> Socket: netty.read (protobuf.response)
Socket -->> Wrapper:
Wrapper ->> Wrapper: Result
end
Wrapper ->> Socket: close()
Wrapper ->> SocketListener: shutdown
SocketListener ->> Socket: close()
deactivate Socket
SocketListener ->> Client: close()
SocketListener -->> Wrapper:
deactivate SocketListener
deactivate worker
deactivate Wrapper
deactivate Client
```

### Elements
* **Wrapper**: Our Babushka wrapper that exposes a client API (java, python, node, etc)
* **Babushka FFI**: Foreign Function Interface definitions from our wrapper to our Rust Babushka-Core
* **Babushka impl**: public interface layer and thread manager
* **Tokio Worker**: Tokio worker threads (number of CPUs)
* **Tokio Worker**: Tokio worker threads (number of CPUs)
* **SocketListener**: listens for work from the Socket, and handles commands
* **Unix Domain Socket**: Unix Domain Socket to handle communication
* **Redis**: Our data store

## Raw-FFI Benchmark Test
## Wrapper-to-Core Connector with raw-FFI calls

**Summary**: We copied the C# benchmarking implementation, and discovered that it wasn't using the Babushka/Redis client
at all, but instead spawning a single worker thread to connect to Redis using a general Rust Redis client.

### (Current) Sequence Diagram for test
**Summary**: Foreign Function Calls are simple to implement inter-language calls. The setup between Golang and the Rust-core
is fairly simple using the well-supported CGO library. While sending language calls is easy, setting it up in an async manner
is more complex because of the requirement to handle async callbacks. Golang as a simple, light-weight solution, using goroutines
and channels, to pass callbacks and execution between langugages.

```mermaid
sequenceDiagram
participant Wrapper as Client-Wrapper
participant ffi as Babushka FFI
participant worker as Tokio Worker
participant Client as Redis
activate Wrapper
activate Client
Wrapper ->>+ ffi: create_connection
ffi ->>+ worker: Create Tokio::Runtime (count: 1)
worker ->> Client: new Redis::Client
ffi -->> Wrapper: Connection
Wrapper ->> ffi: command (GET/SET)
ffi ->>+ worker: Runtime::spawn
worker ->> Client: Connection::clone(command)
Client -->> worker: Result
worker -->> ffi: success_callback
ffi ->> Wrapper: async Result
deactivate Wrapper
deactivate Client
stateDiagram-v2
direction LR
Wrapper: Golang Wrapper
FFI: Foreign Function Interface
RustCore: Rust-Core
[*] --> Wrapper: User
Wrapper --> FFI
FFI --> Wrapper
RustCore --> FFI
FFI --> RustCore
RustCore --> Redis
```

### (New) Sequence Diagram for test
## Decision to use Raw-FFI calls directly to Rust-Core for Golang Wrapper

Shachar [updated C# benchmark tests](https://github.com/aws/babushka/pull/559/files) to connect to babushka core, instead
of a redis client.
### Decision Log

```mermaid
sequenceDiagram
***TODO***

participant Wrapper as Client-Wrapper
participant ffi as Babushka FFI
participant worker as Tokio Worker
participant Client as Redis
| Protocol | Details | Pros | Cons |
|----------------------------------------------|---------|------|------|
| Unix Domain Sockets (jni/netty) | | | |
| Raw-FFI (JNA, uniffi-rs, j4rs, interoptopus) | | | |

activate Wrapper
activate Client
Wrapper ->>+ ffi: create_connection
ffi ->>+ worker: Create Tokio::Runtime (count: 1)
worker ->> Client: BabushkaClient::new
ffi -->> Wrapper: Connection
Wrapper ->> ffi: cmd[GET/SET]
ffi ->>+ worker: Runtime::spawn
worker ->> Client: Connection::clone.req_packed_command(cmd)
Client -->> worker: Result
worker -->> ffi: success_callback
ffi ->> Wrapper: async Result
deactivate Wrapper
deactivate Client
```
## Sequence Diagram - Raw-FFI Client

## Sequence Diagram - Managed Raw-FFI Client
**Summary**: If we make direct calls through FFI from our Wrapper to Rust, we can initiate commands to Redis. This allows us
to make on-demand calls directly to Rust-core solution. Since the calls are async, we need to manage and populate a callback
object with the response and a payload.

**Summary**: Following the socket listener/manager solution, we can create a [event manager](https://en.wikipedia.org/wiki/Reactor_pattern)
on the Rust side that will spawn worker threads available to execute event commands on-demand. FFI calls will petition the
worker thread manager for work request.
We will need to avoid busy waits while waiting on the async response. The wrapper and Rust-core languages independently track
threads. On the Rust side, they use a Tokio runtime to manage threads. When the Rust-core is complete, and returning a Response,
we can use the Callback object to re-awake the wrapper thread manager and continue work.

_Expectation_: According to Shachar, it is our understanding that having a Tokio thead manager on the Rust side, and an event
manager on the Wrapper-side will create a lot of busy-waiting between the two thread managers.
Go routines have a performant solution using light-weight go-routines and channels. Instead of busy-waiting, we awaken by
pushing goroutines to the result channel once the Tokio threads send back a callback.

### Sequence Diagram

_Observation_: Go routines seems to have a decent solution using channels. Instead of waiting, we can close the threads
on the wrapper since, and (awaken) push the threads back to the channel once the Tokio threads are completed.

```mermaid
sequenceDiagram
participant Wrapper as Client-Wrapper
participant channel as Result Channel
participant ffi as Babushka FFI
participant manager as Babushka impl
participant worker as Tokio Worker
participant RuntimeManager as Runtime Manager
participant Client as Redis
activate Wrapper
activate Client
Wrapper ->>+ ffi: connect_to_redis
Wrapper -)+ ffi: create_connection(connection_settings)
ffi ->>+ manager: start_thread_manager(init_callback)
manager ->> worker: Create Tokio::Runtime (count: CPUs)
activate worker
worker ->> RuntimeManager: wait_for_work(init_callback)
RuntimeManager ->> RuntimeManager: loop: wait
activate RuntimeManager
manager -->> ffi: callback
ffi -->>- Wrapper: callback
RuntimeManager -->> Client: BabushkaClient::new
Wrapper ->> ffi: command: get(key)
ffi ->> manager: command: get(key)
manager ->> worker: command: get(key)
worker ->> Client: send(command)
manager -->> Wrapper: Ok(BabushkaClient)
worker ->> Client: BabushkaClient::new
worker ->> worker: wait_for_work(init_callback)
loop single_request
Wrapper ->> channel: make channel
activate channel
Wrapper -) ffi: command: single_command(protobuf.redis_request, &channel)
Wrapper ->> channel: wait
ffi ->> manager: cmd(protobuf.redis_request)
manager ->> worker: command: cmd(protobuf.redis_request)
worker ->> Client: send(command, args)
Client -->> worker: Result
worker -->> ffi: success_callback
ffi ->> Wrapper: async Result
Wrapper ->> ffi: close_connection
ffi ->> manager: close_connection
manager ->>- worker: close
worker ->>RuntimeManager: close
deactivate RuntimeManager
deactivate worker
deactivate Wrapper
deactivate Client
```
worker -->> ffi: Ok(protobuf.response)
ffi -->> channel: Ok(protobuf.response)
channel ->> Wrapper: protobuf.response
Wrapper ->> channel: close
deactivate channel
end
Wrapper -) worker: close_connection
worker -->> Wrapper:
deactivate worker
deactivate Wrapper
deactivate Client
```

### Elements
* **Client-Wrapper**: Our Babushka wrapper that exposes a client API (Go, etc)
* **Result Channel**: Goroutine channel on the Babushka Wrapper
* **Babushka FFI**: Foreign Function Interface definitions from our wrapper to our Rust Babushka-Core
* **Babushka impl**: public interface layer and thread manager
* **Tokio Worker**: Tokio worker threads (number of CPUs)
* **Redis**: Our data store

0 comments on commit 4c30c24

Please sign in to comment.