Merge remote-tracking branch 'upstream/main' into java_benchmarks

Signed-off-by: Yury-Fridlyand <[email protected]>

.github/workflows/java.yml

@@ -1,9 +1,16 @@
-name: SQL Java CI
+name: Java CI
 
 on:
-  pull_request:
   push:
     paths:
+      - babushka-core/**
+      - submodules/**
+      - "java/**"
+      - ".github/workflows/java.yml"
+  pull_request:
+    paths:
+      - babushka-core/**
+      - submodules/**
       - "java/**"
       - ".github/workflows/java.yml"
 

.github/workflows/python.yml

@@ -51,7 +51,7 @@ jobs:
               working-directory: ./python
               run: |
                   python -m pip install --upgrade pip
-                  pip install flake8 isort black mypy-protobuf
+                  pip install flake8 isort black mypy-protobuf deadcode
 
             - name: Lint with isort
               working-directory: ./python
@@ -71,6 +71,11 @@ jobs:
               run: |
                   black --target-version py36 --check --diff .
 
+            - name: Lint with deadcode
+              working-directory: .
+              run: |
+                  deadcode . --ignore-definitions=InfoSection,Level,*protobuf*,*pytest*
+
             - name: Start redis server
               working-directory: ./python
               run: redis-server &

CHANGELOG.md

@@ -0,0 +1,5 @@
+## 0.0.1
+
+Preview release of **babushka** a Polyglot Redis client.
+
+See the [README](README.md) for additional information.

CODEOWNERS

@@ -0,0 +1 @@
+* @aws/babushka

CONTRIBUTING.md

@@ -0,0 +1,62 @@
+# Contributing Guidelines
+
+Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
+documentation, we greatly value feedback and contributions from our community.
+
+Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
+information to effectively respond to your bug report or contribution.
+
+
+## Reporting Bugs/Feature Requests
+
+We welcome you to use the GitHub issue tracker to report bugs or suggest features.
+
+When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
+reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
+
+* A reproducible test case or series of steps
+* The version of our code being used
+* Operating system
+* Redis version
+* Redis cluster information, cluster topology, number of shards, number of replicas, used data types 
+* Any modifications you've made relevant to the bug
+* Anything unusual about your environment or deployment
+* logs
+
+## Contributing via Pull Requests
+Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
+
+1. You are working against the latest source on the *main* branch.
+2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+
+To send us a pull request, please:
+
+1. Fork the repository.
+2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
+3. Ensure local tests pass.
+4. Commit to your fork using clear commit messages.
+5. Send us a pull request, answering any default questions in the pull request interface.
+6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+
+GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
+[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
+
+
+## Finding contributions to work on
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
+
+
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
+[email protected] with any additional questions or comments.
+
+
+## Security issue notifications
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+
+
+## Licensing
+
+See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.

SECURITY.md

@@ -0,0 +1,11 @@
+## Reporting Security Issues
+
+We take all security reports seriously.
+When we receive such reports,
+we will investigate and subsequently address
+any potential vulnerabilities as quickly as possible.
+If you discover a potential security issue in this project,
+please notify AWS/Amazon Security via our
+[vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/)
+or directly via email to [AWS Security](mailto:[email protected]).
+Please do *not* create a public GitHub issue in this project.

babushka-core/src/protobuf/redis_request.proto

@@ -73,6 +73,29 @@ enum RequestType {
     Decr=35;
     IncrByFloat=36;
     DecrBy=37;
+    HashGetAll=38;
+    HashMSet=39;
+    HashMGet=40;
+    HashIncrBy = 41;
+    HashIncrByFloat = 42;
+    LPush = 43;
+    LPop = 44;
+    RPush = 45;
+    RPop = 46;
+    LLen = 47;
+    LRem = 48;
+    LRange = 49;
+    LTrim = 50;
+    SAdd = 51;
+    SRem = 52;
+    SMembers = 53;
+    SCard = 54;
+    PExpireAt = 55;
+    PExpire = 56;
+    ExpireAt = 57;
+    Exists = 58;
+    Unlink = 59;
+    TTL = 60;
 }
 
 message Command {

babushka-core/src/socket_listener.rs

@@ -308,6 +308,29 @@ fn get_command(request: &Command) -> Option<Cmd> {
         RequestType::IncrByFloat => Some(cmd("INCRBYFLOAT")),
         RequestType::Decr => Some(cmd("DECR")),
         RequestType::DecrBy => Some(cmd("DECRBY")),
+        RequestType::HashGetAll => Some(cmd("HGETALL")),
+        RequestType::HashMSet => Some(cmd("HMSET")),
+        RequestType::HashMGet => Some(cmd("HMGET")),
+        RequestType::HashIncrBy => Some(cmd("HINCRBY")),
+        RequestType::HashIncrByFloat => Some(cmd("HINCRBYFLOAT")),
+        RequestType::LPush => Some(cmd("LPUSH")),
+        RequestType::LPop => Some(cmd("LPOP")),
+        RequestType::RPush => Some(cmd("RPUSH")),
+        RequestType::RPop => Some(cmd("RPOP")),
+        RequestType::LLen => Some(cmd("LLEN")),
+        RequestType::LRem => Some(cmd("LREM")),
+        RequestType::LRange => Some(cmd("LRANGE")),
+        RequestType::LTrim => Some(cmd("LTRIM")),
+        RequestType::SAdd => Some(cmd("SADD")),
+        RequestType::SRem => Some(cmd("SREM")),
+        RequestType::SMembers => Some(cmd("SMEMBERS")),
+        RequestType::SCard => Some(cmd("SCARD")),
+        RequestType::PExpireAt => Some(cmd("PEXPIREAT")),
+        RequestType::PExpire => Some(cmd("PEXPIRE")),
+        RequestType::ExpireAt => Some(cmd("EXPIREAT")),
+        RequestType::Exists => Some(cmd("EXISTS")),
+        RequestType::Unlink => Some(cmd("UNLINK")),
+        RequestType::TTL => Some(cmd("TTL")),
     }
 }
 

benchmarks/utilities/fill_db.ts

@@ -20,7 +20,7 @@ async function fill_database(
     const sets = Array.from(Array(CONCURRENT_SETS).keys()).map(
         async (index) => {
             for (let i = 0; i < SIZE_SET_KEYSPACE / CONCURRENT_SETS; ++i) {
-                const key = (index * CONCURRENT_SETS + index).toString();
+                const key = (i * CONCURRENT_SETS + index).toString();
                 await client.set(key, data);
             }
         }

examples/python/client_example.py

@@ -73,9 +73,9 @@ async def test_cluster_client(host: str = "localhost", port: int = 6379):
 
 async def main():
     set_console_logger(LogLevel.DEBUG)
-    # set_file_logger(LogLevel.DEBUG)
+    set_file_logger(LogLevel.DEBUG)
     await test_standalone_client()
-    # await test_cluster_client()
+    await test_cluster_client()
 
 
 if __name__ == "__main__":

java/.cargo/config.toml

@@ -1,3 +1,3 @@
 [env]
-BABUSHKA_NAME = { value = "BabushkaPy", force = true }
+BABUSHKA_NAME = { value = "javababushka", force = true }
 BABUSHKA_VERSION = "0.1.0"

java/README.md

@@ -8,9 +8,9 @@ language api to enhance performance and limit cpu cycles at scale.
 
 The Java client (javababushka) contains the following parts:
 
-1. A Java client (lib folder): wrapper to rust-client
-2. An examples script: to sanity test javababushka and similar java-clients against a redis host
-3. A benchmark app: to performance benchmark test javababushka and similar java-clients against a redis host
+1. A Java client (lib folder): wrapper to rust-client.
+2. An examples script: to sanity test javababushka and similar java-clients against a redis host.
+3. A benchmark app: to performance benchmark test javababushka and similar java-clients against a redis host.
 
 ## Building
 
@@ -29,20 +29,21 @@ These IDE plugins can auto-format code on file save or by single click:
 You can run benchmarks using `./gradlew run`. You can set arguments using the args flag like:
 
 ```shell
-./gradlew run --args="--clients lettuce"
+./gradlew run --args="-help"
+./gradlew run --args="-resultsFile=output.json -dataSize \"100 1000\" -concurrentTasks \"10 100\" -clients all -host localhost -port 6279 -clientCount \"1 5\" -tls"
 ```
 
 The following arguments are accepted: 
-* `configuration`: Release or Debug configuration
 * `resultsFile`: the results output file
-* `concurrentTasks`: Number of concurrent tasks 
+* `concurrentTasks`: Number of concurrent tasks
 * `clients`: one of: all|jedis|lettuce|babushka
 * `clientCount`: Client count
-* `host`: redis server host url 
+* `host`: redis server host url
 * `port`: redis server port number
 * `tls`: redis TLS configured
 
 ### Troubleshooting
 
-* If you're unable to connect to redis (such as timeout), check your port or the TLS flag
-* Only server-side certificates are supported by the TLS configured redis
+* Connection Timeout:
+  * If you're unable to connect to redis, check that you are connecting to the correct host, port, and TLS configuration.
+* Only server-side certificates are supported by the TLS configured redis.