Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split netty client into two files #40

Closed
Closed
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
06574ea
Add Jedis and Lettuce benchmarks
jonathanl-bq Sep 7, 2023
5990767
Start ignoring .gradle files
jonathanl-bq Sep 8, 2023
6ae93f5
Update gitignore and remove generated files from git
acarbonetto Oct 3, 2023
8203c4d
Update gitignore and remove generated files from git
acarbonetto Sep 8, 2023
e517744
Update gitignore and remove generated files from git
acarbonetto Sep 8, 2023
2a11e9a
Add benchmarks for GET non-existing
jonathanl-bq Sep 8, 2023
cbb0dcb
Revert "Update gitignore and remove generated files from git"
acarbonetto Sep 8, 2023
5f51a5b
fix redis-rs submodules
acarbonetto Sep 8, 2023
52df672
Randomize commands in Java benchmarks
jonathanl-bq Sep 13, 2023
84f0efc
rename chooseAction to randomAction
jonathanl-bq Sep 14, 2023
1983974
Add a Java benchmarking app (#7)
acarbonetto Sep 20, 2023
6c1fb45
Add Readme and update install_and_test script to runJava
acarbonetto Sep 20, 2023
46d0cf6
Add Readme and update install_and_test script to runJava
acarbonetto Sep 20, 2023
d526f96
Combine java pipeline and java benchmarks (#8)
Yury-Fridlyand Sep 22, 2023
436da8f
Add sync and async clients both to tests. (#12)
Yury-Fridlyand Sep 25, 2023
016f5f6
Move duplicated logic in benchmark JS scripts to a single file, and c…
acarbonetto Sep 25, 2023
e57c1ff
Add dataSize option to java benchmark. (#11)
Yury-Fridlyand Sep 26, 2023
8664d05
Rename jabushka to javababushka (#14)
acarbonetto Sep 26, 2023
b15f93e
Add missing renames. (#17)
Yury-Fridlyand Sep 27, 2023
1bab56a
Add option to run tests on multiple clients in concurrency (#16)
Yury-Fridlyand Oct 5, 2023
d99d27a
Update redis-rs to match main branch
acarbonetto Oct 6, 2023
540f49a
Create clients only once per iteration (#19)
acarbonetto Oct 6, 2023
035ef20
Add Java-client benchmarking app
acarbonetto Oct 10, 2023
0883985
spotless apply
acarbonetto Oct 10, 2023
d15ade1
Update on command line options
acarbonetto Oct 10, 2023
e3634ba
Update README
acarbonetto Oct 10, 2023
500a043
Spotless apply:
acarbonetto Oct 10, 2023
0bf1678
Update README example
acarbonetto Oct 11, 2023
12efc63
update commandline defaults for review comments
acarbonetto Oct 11, 2023
951aa8b
Remove TLS flag argument from option
acarbonetto Oct 11, 2023
adc0bb6
Add lettuce clients for benchmarking
acarbonetto Oct 10, 2023
9fc9022
Spotless apply
acarbonetto Oct 10, 2023
111fc26
Add Jedis clients
acarbonetto Oct 10, 2023
9236e9a
Add to app
acarbonetto Oct 10, 2023
49654c3
Add for-loop for data size list
acarbonetto Oct 10, 2023
b3284c1
Add TPS for all async items
acarbonetto Oct 11, 2023
8f1dfa5
spotless apply
acarbonetto Oct 11, 2023
7f690cf
Fix TPS calculations
acarbonetto Oct 11, 2023
d00a655
Accept TLS as a flag
acarbonetto Oct 11, 2023
b5e9a33
Merge branch 'java-client/add_benchmark_app' into java-client/add_jed…
acarbonetto Oct 11, 2023
3cee673
Merge branch 'java-client/add_jedis_client' into java-client/add_tps
acarbonetto Oct 11, 2023
8dcfec0
Start threads; then wait for results
acarbonetto Oct 12, 2023
a615f8a
Add java-jni client
acarbonetto Oct 18, 2023
73c448f
Handle Exceptions from client; add JniSyncClient fixes
acarbonetto Oct 19, 2023
79459f5
Clean up latency and add error checking
acarbonetto Oct 20, 2023
c3d235a
Add json reporting. (#10)
Yury-Fridlyand Oct 20, 2023
917dd0e
Merge remote-tracking branch 'origin/java_benchmarks' into java-clien…
Yury-Fridlyand Oct 24, 2023
9e62324
Minor fixes.
Yury-Fridlyand Oct 24, 2023
11a78a3
Fix result printing.
Yury-Fridlyand Oct 25, 2023
a2c2670
Add TPS.
Yury-Fridlyand Oct 25, 2023
4b9f8de
Remove duplicates. Reorganize and fix imports.
Yury-Fridlyand Oct 25, 2023
d6b72ac
Int ctor fix.
Yury-Fridlyand Oct 26, 2023
58f7b58
Iteration 1.
Yury-Fridlyand Oct 31, 2023
d145bdf
Iteration 2: connected!
Yury-Fridlyand Oct 31, 2023
0b03dc5
Iteration 3: `get` and `set`.
Yury-Fridlyand Nov 1, 2023
a3075d5
Iteration 4: benchmark.
Yury-Fridlyand Nov 1, 2023
5f10964
Iteration 5: some fixes.
Yury-Fridlyand Nov 2, 2023
e3f7596
Change number of threads in Benchmarking threadpool
jonathanl-bq Nov 2, 2023
49c9119
Revert "Change number of threads in Benchmarking threadpool"
jonathanl-bq Nov 2, 2023
dd7413b
Add more flushing rules and UT.
Yury-Fridlyand Nov 3, 2023
a70c907
Client clean up.
Yury-Fridlyand Nov 16, 2023
14d392b
try to split client - receiving unknown callback id
Yury-Fridlyand Nov 16, 2023
058a05c
Iteration 2 - try to use a singleton-like approach.
Yury-Fridlyand Nov 20, 2023
dfbd106
Fix thread pool use for Netty.
Yury-Fridlyand Nov 24, 2023
ea30ea1
Revert some core lib changes.
Yury-Fridlyand Nov 24, 2023
9edb6be
Merge remote-tracking branch 'origin/integ-java-client-milestone-1' i…
Yury-Fridlyand Nov 24, 2023
986f468
Revert some changes added by mistake.
Yury-Fridlyand Nov 24, 2023
d58bff9
Address PR feedback.
Yury-Fridlyand Nov 27, 2023
507ef9f
Fix generating callback ID and handling new connections.
Yury-Fridlyand Nov 27, 2023
9cecf23
Revert IT.
Yury-Fridlyand Nov 28, 2023
8ffdbea
Add unit tests for Client
acarbonetto Nov 29, 2023
4ecd5d8
Implement Bulk value case in FFI layer
jonathanl-bq Nov 29, 2023
2bf997e
More split.
Yury-Fridlyand Nov 30, 2023
1aa0707
Renaming
Yury-Fridlyand Nov 30, 2023
14e3e18
Fix build.
Yury-Fridlyand Nov 30, 2023
6d2ae81
Clean up commented logging.
Yury-Fridlyand Nov 30, 2023
e7ee9a9
Javadoc
Yury-Fridlyand Nov 30, 2023
540120e
Revert some renamings.
Yury-Fridlyand Nov 30, 2023
aa5aa52
Optimize a bit.
Yury-Fridlyand Nov 30, 2023
9a0c5d6
Split more.
Yury-Fridlyand Dec 1, 2023
2c9aa38
Grab some changes from upstream.
Yury-Fridlyand Dec 1, 2023
fbff211
Misc fixes.
Yury-Fridlyand Dec 2, 2023
701da44
Open channel (connection) per client, share thread pools.
Yury-Fridlyand Dec 4, 2023
5bf6672
Minor fixes.
Yury-Fridlyand Dec 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions babushka-core/src/socket_listener.rs
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ async fn write_result(
error_message.into(),
))
} else {
log_warn("received error", error_message.as_str());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you mean to check this in?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is more verbose

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While true, this change is irrelevant to the PR. We should put this in a separate PR.

log_warn("received error", format!("{} for callback {}", error_message.as_str(), callback_index));
let mut request_error = response::RequestError::default();
if err.is_connection_dropped() {
request_error.type_ = response::RequestErrorType::Disconnect.into();
Expand Down Expand Up @@ -476,7 +476,7 @@ fn handle_request(request: RedisRequest, client: Client, writer: Rc<Writer>) {
}
},
None => Err(ClienUsageError::InternalError(
"Received empty request".to_string(),
format!("Received empty request for callback {}", request.callback_idx),
)),
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,6 @@ public String getName() {
return name;
}

@Override
public void closeConnection() {
testClient.closeConnection();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we not need to close the connection anymore? I'm not sure why this is removed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another client instance may still use the connection - it is common for all.

}

@Override
public void connectToRedis() {
connectToRedis(new ConnectionSettings("localhost", 6379, false, false));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ public static void printResults(
System.out.printf("Iterations: %d%n", iterations);
System.out.printf("TPS: %f%n", iterations / duration);
int totalHits = 0;
acarbonetto marked this conversation as resolved.
Show resolved Hide resolved

for (Map.Entry<ChosenAction, LatencyResults> entry : resultsMap.entrySet()) {
ChosenAction action = entry.getKey();
LatencyResults results = entry.getValue();
Expand All @@ -140,7 +141,7 @@ public static void printResults(
System.out.printf("Total hits: %d%n", results.totalHits);
totalHits += results.totalHits;
}
System.out.println("Total hits: " + totalHits);
System.out.printf("%nTotal hits: %d%n", totalHits);
}

public static void testClientSetGet(
Expand Down
7 changes: 7 additions & 0 deletions java/client/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ dependencies {
implementation group: 'io.netty', name: 'netty-transport-native-epoll', version: '4.1.100.Final', classifier: 'linux-x86_64'
implementation group: 'io.netty', name: 'netty-transport-native-kqueue', version: '4.1.100.Final', classifier: 'osx-x86_64'
implementation group: 'io.netty', name: 'netty-transport-native-kqueue', version: '4.1.100.Final', classifier: 'osx-aarch_64'

//lombok
compileOnly 'org.projectlombok:lombok:1.18.30'
annotationProcessor 'org.projectlombok:lombok:1.18.30'
testCompileOnly 'org.projectlombok:lombok:1.18.30'
testAnnotationProcessor 'org.projectlombok:lombok:1.18.30'
}

tasks.register('protobuf', Exec) {
Expand Down Expand Up @@ -60,6 +66,7 @@ tasks.register('buildAll') {
}

compileJava.dependsOn('protobuf')

clean.dependsOn('cleanProtobuf')

tasks.withType(Test) {
Expand Down
Loading
Loading