diff --git a/docs/design-raw-ffi.md b/docs/design-raw-ffi.md index 331651613a..10711a8b6a 100644 --- a/docs/design-raw-ffi.md +++ b/docs/design-raw-ffi.md @@ -1,16 +1,58 @@ -# Babushka Socket Listener +# Babushka Core Wrappers -## Unix Domain Socket Manager +## Summary -### Sequence Diagram +The Babushka client enables Redis users connect to Redis using a variety of languages and supported commands. The client +uses a performant core to establish and manage connections, and a thin wrapper layer written in various languages, to +communicate -**Summary**: The Babushka "UDS" solution uses a socket manager to redis-client worker threads, and UDS to manage the communication -between the wrapper and redis-client threads. This works well because we allow the socket to manage the communication. This -results in simple/fast communication. But the worry is that the unix sockets can become a bottleneck for data-intensive communication; -ie read/write buffer operations become the bottleneck. +The following document discusses two primary communication protocol architectures for wrapping the babushka clients, and +how Java-Babushka and Go-Babushka use each of those protocols differently by taking advantage of language-specific attributes. -_Observation_: We noticed that we are creating a fixed number of Babushka/Redis client connections based on the number of CPUs available. -It would be better to configure this thread count, or default it to the number of CPUs available. +### References + +* Babushka - communication protocol architecture: describes high-level approaches to communication protocols. + +# Unix Domain Socket Manager Connector + +## High-Level Design + +**Summary**: The Babushka "UDS" solution uses a socket listener to manage rust-to-wrapper worker threads, and unix domain sockets +to deliver package between the wrapper and redis-client threads. This works well because we allow the unix sockets to pass messages +through the OS and is very performant. This results in simple/fast communication. But the concern is that the unix sockets can +become a bottleneck for data-intensive communication; ie read/write buffer operations. + +```mermaid +stateDiagram-v2 + direction LR + + JavaWrapper: Java Wrapper + UnixDomainSocket: Unix Domain Socket + RustCore: Rust-Core + + [*] --> JavaWrapper: User + JavaWrapper --> UnixDomainSocket + UnixDomainSocket --> JavaWrapper + RustCore --> UnixDomainSocket + UnixDomainSocket --> RustCore + RustCore --> Redis +``` + +## Decision to use UDS Sockets for a Java-Babushka Wrapper + +The decision to use Unix Domain Sockets (UDS) to manage the Java-wrapper to Babushka Redis-client communication was two-fold, as opposed to a raw-FFI solution: +1. Java contains an efficient socket protocol library (netty.io) that provides a highly configurable environment to manage sockets. +2. Java objects serialization/de-serialization is an expensive operation, and a performing multiple io operations between raw-ffi calls would be inefficient. + +### Decision Log + +| Protocol | Details | Pros | Cons | +|----------------------------------------------|---------------------------------------------------------|-----------------------------|----------------------------------------------------| +| Unix Domain Sockets (jni/netty) | JNI to submit commands; netty.io for message passing | netty.io standard lib; | complex configuration; limited by socket interface | +| Raw-FFI (JNA, uniffi-rs, j4rs, interoptopus) | JNA to submit commands; protobuf for message processing | reusable in other languages | Slow performance and uses JNI under the hood; | +| Panama/jextract | Performance similar to a raw-ffi using JNI | | lacks early Java support (JDK 18+); prototype | + +### Sequence Diagram ```mermaid sequenceDiagram @@ -24,10 +66,9 @@ participant Socket as Unix Domain Socket participant Client as Redis activate Wrapper -activate Socket activate Client -Wrapper ->>+ ffi: connect_to_redis -ffi ->>+ manager: start_socket_listener(init_callback) +Wrapper -)+ ffi: connect_to_redis +ffi -)+ manager: start_socket_listener(init_callback) manager ->> worker: Create Tokio::Runtime (count: CPUs) activate worker worker ->> SocketListener: listen_on_socket(init_callback) @@ -35,149 +76,144 @@ ffi ->>+ manager: start_socket_listener(init_callback) activate SocketListener manager -->> ffi: socket_path ffi -->>- Wrapper: socket_path - SocketListener -->> Client: BabushkaClient::new - SocketListener -->> Socket: UnixStreamListener::new -Wrapper ->> Socket: write buffer (socket_path) -Socket -->> Wrapper: - SocketListener ->> SocketListener: handle_request - SocketListener ->> Socket: read_values_loop(client_listener, client) - Socket -->> SocketListener: - SocketListener ->> Client: send(request) - Client -->> SocketListener: ClientUsageResult - SocketListener ->> Socket: write_result - Socket -->> SocketListener: -Wrapper ->> Socket: read buffer (socket_path) + SocketListener -->> Socket: UnixStreamListener::new + activate Socket + SocketListener -->> Client: BabushkaClient::new +Wrapper ->> Socket: connect + Socket -->> Wrapper: +loop single_request + Wrapper ->> Socket: netty.writeandflush (protobuf.redis_request) Socket -->> Wrapper: -Wrapper ->> Wrapper: Result -Wrapper ->> ffi: close_connection - ffi ->> manager: close_connection - manager ->>- worker: close - worker ->>SocketListener: close - deactivate SocketListener - deactivate worker -deactivate Wrapper -deactivate Client -deactivate Socket + Wrapper ->> Wrapper: wait + SocketListener ->> SocketListener: handle_request + SocketListener ->> Socket: read_values_loop(client_listener, client) + Socket -->> SocketListener: + SocketListener ->> Client: send(request) + Client -->> SocketListener: ClientUsageResult + SocketListener ->> Socket: write_result + Socket -->> SocketListener: + Wrapper ->> Socket: netty.read (protobuf.response) + Socket -->> Wrapper: + Wrapper ->> Wrapper: Result +end +Wrapper ->> Socket: close() +Wrapper ->> SocketListener: shutdown + SocketListener ->> Socket: close() + deactivate Socket + SocketListener ->> Client: close() + SocketListener -->> Wrapper: + deactivate SocketListener + deactivate worker + deactivate Wrapper + deactivate Client ``` ### Elements * **Wrapper**: Our Babushka wrapper that exposes a client API (java, python, node, etc) * **Babushka FFI**: Foreign Function Interface definitions from our wrapper to our Rust Babushka-Core * **Babushka impl**: public interface layer and thread manager -* **Tokio Worker**: Tokio worker threads (number of CPUs) +* **Tokio Worker**: Tokio worker threads (number of CPUs) * **SocketListener**: listens for work from the Socket, and handles commands * **Unix Domain Socket**: Unix Domain Socket to handle communication * **Redis**: Our data store -## Raw-FFI Benchmark Test +## Wrapper-to-Core Connector with raw-FFI calls -**Summary**: We copied the C# benchmarking implementation, and discovered that it wasn't using the Babushka/Redis client -at all, but instead spawning a single worker thread to connect to Redis using a general Rust Redis client. - -### (Current) Sequence Diagram for test +**Summary**: Foreign Function Calls are simple to implement inter-language calls. The setup between Golang and the Rust-core +is fairly simple using the well-supported CGO library. While sending language calls is easy, setting it up in an async manner +is more complex because of the requirement to handle async callbacks. Golang as a simple, light-weight solution, using goroutines +and channels, to pass callbacks and execution between langugages. ```mermaid -sequenceDiagram - -participant Wrapper as Client-Wrapper -participant ffi as Babushka FFI -participant worker as Tokio Worker -participant Client as Redis - -activate Wrapper -activate Client -Wrapper ->>+ ffi: create_connection - ffi ->>+ worker: Create Tokio::Runtime (count: 1) - worker ->> Client: new Redis::Client - ffi -->> Wrapper: Connection -Wrapper ->> ffi: command (GET/SET) - ffi ->>+ worker: Runtime::spawn - worker ->> Client: Connection::clone(command) - Client -->> worker: Result - worker -->> ffi: success_callback -ffi ->> Wrapper: async Result -deactivate Wrapper -deactivate Client +stateDiagram-v2 + direction LR + + Wrapper: Golang Wrapper + FFI: Foreign Function Interface + RustCore: Rust-Core + + [*] --> Wrapper: User + Wrapper --> FFI + FFI --> Wrapper + RustCore --> FFI + FFI --> RustCore + RustCore --> Redis ``` -### (New) Sequence Diagram for test +## Decision to use Raw-FFI calls directly to Rust-Core for Golang Wrapper -Shachar [updated C# benchmark tests](https://github.com/aws/babushka/pull/559/files) to connect to babushka core, instead -of a redis client. +### Decision Log -```mermaid -sequenceDiagram +***TODO*** -participant Wrapper as Client-Wrapper -participant ffi as Babushka FFI -participant worker as Tokio Worker -participant Client as Redis +| Protocol | Details | Pros | Cons | +|----------------------------------------------|---------|------|------| +| Unix Domain Sockets (jni/netty) | | | | +| Raw-FFI (JNA, uniffi-rs, j4rs, interoptopus) | | | | -activate Wrapper -activate Client -Wrapper ->>+ ffi: create_connection - ffi ->>+ worker: Create Tokio::Runtime (count: 1) - worker ->> Client: BabushkaClient::new - ffi -->> Wrapper: Connection -Wrapper ->> ffi: cmd[GET/SET] - ffi ->>+ worker: Runtime::spawn - worker ->> Client: Connection::clone.req_packed_command(cmd) - Client -->> worker: Result - worker -->> ffi: success_callback -ffi ->> Wrapper: async Result -deactivate Wrapper -deactivate Client -``` +## Sequence Diagram - Raw-FFI Client -## Sequence Diagram - Managed Raw-FFI Client +**Summary**: If we make direct calls through FFI from our Wrapper to Rust, we can initiate commands to Redis. This allows us +to make on-demand calls directly to Rust-core solution. Since the calls are async, we need to manage and populate a callback +object with the response and a payload. -**Summary**: Following the socket listener/manager solution, we can create a [event manager](https://en.wikipedia.org/wiki/Reactor_pattern) -on the Rust side that will spawn worker threads available to execute event commands on-demand. FFI calls will petition the -worker thread manager for work request. +We will need to avoid busy waits while waiting on the async response. The wrapper and Rust-core languages independently track +threads. On the Rust side, they use a Tokio runtime to manage threads. When the Rust-core is complete, and returning a Response, +we can use the Callback object to re-awake the wrapper thread manager and continue work. -_Expectation_: According to Shachar, it is our understanding that having a Tokio thead manager on the Rust side, and an event -manager on the Wrapper-side will create a lot of busy-waiting between the two thread managers. +Go routines have a performant solution using light-weight go-routines and channels. Instead of busy-waiting, we awaken by +pushing goroutines to the result channel once the Tokio threads send back a callback. + +### Sequence Diagram -_Observation_: Go routines seems to have a decent solution using channels. Instead of waiting, we can close the threads -on the wrapper since, and (awaken) push the threads back to the channel once the Tokio threads are completed. ```mermaid sequenceDiagram participant Wrapper as Client-Wrapper +participant channel as Result Channel participant ffi as Babushka FFI participant manager as Babushka impl participant worker as Tokio Worker -participant RuntimeManager as Runtime Manager participant Client as Redis activate Wrapper activate Client -Wrapper ->>+ ffi: connect_to_redis +Wrapper -)+ ffi: create_connection(connection_settings) ffi ->>+ manager: start_thread_manager(init_callback) manager ->> worker: Create Tokio::Runtime (count: CPUs) activate worker - worker ->> RuntimeManager: wait_for_work(init_callback) - RuntimeManager ->> RuntimeManager: loop: wait - activate RuntimeManager - manager -->> ffi: callback -ffi -->>- Wrapper: callback - RuntimeManager -->> Client: BabushkaClient::new - -Wrapper ->> ffi: command: get(key) - ffi ->> manager: command: get(key) - manager ->> worker: command: get(key) - worker ->> Client: send(command) + manager -->> Wrapper: Ok(BabushkaClient) + worker ->> Client: BabushkaClient::new + worker ->> worker: wait_for_work(init_callback) + +loop single_request +Wrapper ->> channel: make channel + activate channel +Wrapper -) ffi: command: single_command(protobuf.redis_request, &channel) +Wrapper ->> channel: wait + ffi ->> manager: cmd(protobuf.redis_request) + manager ->> worker: command: cmd(protobuf.redis_request) + worker ->> Client: send(command, args) Client -->> worker: Result - worker -->> ffi: success_callback -ffi ->> Wrapper: async Result - -Wrapper ->> ffi: close_connection - ffi ->> manager: close_connection - manager ->>- worker: close - worker ->>RuntimeManager: close - deactivate RuntimeManager - deactivate worker -deactivate Wrapper -deactivate Client -``` \ No newline at end of file + worker -->> ffi: Ok(protobuf.response) + ffi -->> channel: Ok(protobuf.response) +channel ->> Wrapper: protobuf.response +Wrapper ->> channel: close + deactivate channel +end + +Wrapper -) worker: close_connection + worker -->> Wrapper: + deactivate worker + deactivate Wrapper + deactivate Client +``` + +### Elements +* **Client-Wrapper**: Our Babushka wrapper that exposes a client API (Go, etc) +* **Result Channel**: Goroutine channel on the Babushka Wrapper +* **Babushka FFI**: Foreign Function Interface definitions from our wrapper to our Rust Babushka-Core +* **Babushka impl**: public interface layer and thread manager +* **Tokio Worker**: Tokio worker threads (number of CPUs) +* **Redis**: Our data store \ No newline at end of file