Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add uniffi-Kotlin-java-sync-client to java benchmarking #25

Closed
wants to merge 4 commits into from
Closed

Add uniffi-Kotlin-java-sync-client to java benchmarking #25

wants to merge 4 commits into from

Conversation

Yury-Fridlyand
Copy link

Please, ignore some unrelated changes caused by merging independent dev branches. This PR is not supposed be merged. It is a discussion thread.

Code it this PR is based on #23.

Rust-Kotlin code is not cleaned up. The working functions are:

  • connect
  • get3

Note: kotlin client measurements for set are omitted.

Config Action Kotlin Jedis Lettuce
1 client 10 concurrent Get NE p50 0.077 0.024 0.045
1 client 10 concurrent Get NE p90 0.088 0.027 0.054
1 client 10 concurrent Get NE p99 0.161 0.037 0.089
1 client 10 concurrent Get NE avg 0.098 0.025 0.049
1 client 10 concurrent Get NE std 2.276 0.004 0.020
1 client 10 concurrent Get E p50 0.078 0.024 0.046
1 client 10 concurrent Get E p90 0.088 0.027 0.055
1 client 10 concurrent Get E p99 0.166 0.037 0.094
1 client 10 concurrent Get E avg 0.094 0.025 0.050
1 client 10 concurrent Get E std 1.807 0.004 0.135
1 client 10 concurrent Set p50 0.025 0.048
1 client 10 concurrent Set p90 0.030 0.058
1 client 10 concurrent Set p99 0.041 0.105
1 client 10 concurrent Set avg 0.027 0.052
1 client 10 concurrent Set std 0.007 0.035
2 client 10 concurrent Get NE p50 0.080 0.024 0.045
2 client 10 concurrent Get NE p90 0.091 0.025 0.050
2 client 10 concurrent Get NE p99 0.104 0.032 0.065
2 client 10 concurrent Get NE avg 0.090 0.024 0.046
2 client 10 concurrent Get NE std 1.039 0.002 0.004
2 client 10 concurrent Get E p50 0.080 0.024 0.045
2 client 10 concurrent Get E p90 0.092 0.026 0.051
2 client 10 concurrent Get E p99 0.104 0.032 0.065
2 client 10 concurrent Get E avg 0.095 0.025 0.046
2 client 10 concurrent Get E std 1.394 0.002 0.005
2 client 10 concurrent Set p50 0.025 0.046
2 client 10 concurrent Set p90 0.028 0.052
2 client 10 concurrent Set p99 0.034 0.068
2 client 10 concurrent Set avg 0.026 0.047
2 client 10 concurrent Set std 0.010 0.005
1 client 100 concurrent Get NE p50 0.076 0.024 0.044
1 client 100 concurrent Get NE p90 0.084 0.026 0.047
1 client 100 concurrent Get NE p99 0.097 0.032 0.056
1 client 100 concurrent Get NE avg 0.095 0.025 0.045
1 client 100 concurrent Get NE std 1.840 0.005 0.020
1 client 100 concurrent Get E p50 0.076 0.024 0.045
1 client 100 concurrent Get E p90 0.085 0.026 0.049
1 client 100 concurrent Get E p99 0.097 0.032 0.058
1 client 100 concurrent Get E avg 0.089 0.014 0.046
1 client 100 concurrent Get E std 1.425 0.025 0.003
1 client 100 concurrent Set p50 0.025 0.044
1 client 100 concurrent Set p90 0.028 0.047
1 client 100 concurrent Set p99 0.034 0.056
1 client 100 concurrent Set avg 0.026 0.045
1 client 100 concurrent Set std 0.006 0.003
2 client 100 concurrent Get NE p50 0.079 0.024 0.044
2 client 100 concurrent Get NE p90 0.084 0.025 0.047
2 client 100 concurrent Get NE p99 0.099 0.032 0.056
2 client 100 concurrent Get NE avg 0.085 0.024 0.045
2 client 100 concurrent Get NE std 1.128 0.007 0.004
2 client 100 concurrent Get E p50 0.079 0.024 0.045
2 client 100 concurrent Get E p90 0.085 0.025 0.049
2 client 100 concurrent Get E p99 0.099 0.032 0.058
2 client 100 concurrent Get E avg 0.092 0.025 0.046
2 client 100 concurrent Get E std 1.629 0.009 0.003
2 client 100 concurrent Set p50 0.025 0.044
2 client 100 concurrent Set p90 0.026 0.048
2 client 100 concurrent Set p99 0.033 0.056
2 client 100 concurrent Set avg 0.025 0.045
2 client 100 concurrent Set std 0.017 0.003

To re-run benchmark:

  1. Check out uniffi-kotlin-java-bench branch
  2. Run ./gradlew build in kotlin dir
  3. Run ./gradlew :benchmarks:run in java dir
    (if you're using a remote cluster, add --args="<redis-cli args>" to gradle command line, e.g. ./gradlew :benchmarks:run --args="--tls -h redis-cloud.aws.com")

(considering you already have java 17 installed sudo apt install openjdk-17-jre-headless)

Plain log 
 =====> Jedis <===== 1 clients 10 concurrent

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Avg. time in ms per GET_NON_EXISTING: 0.025610913327061776
GET_NON_EXISTING p50 latency in ms: 0.024761
GET_NON_EXISTING p90 latency in ms: 0.027838
GET_NON_EXISTING p99 latency in ms: 0.037822
GET_NON_EXISTING std dev in ms: 0.004191647190735822
Avg. time in ms per SET: 0.02710067883101151
SET p50 latency in ms: 0.025704
SET p90 latency in ms: 0.030322
SET p99 latency in ms: 0.041017
SET std dev in ms: 0.007741070929232292
Avg. time in ms per GET_EXISTING: 0.02560879077876189
GET_EXISTING p50 latency in ms: 0.024673
GET_EXISTING p90 latency in ms: 0.027759
GET_EXISTING p99 latency in ms: 0.037345
GET_EXISTING std dev in ms: 0.00498995224628289

 =====> Jedis <===== 2 clients 10 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.024919999563563812
GET_NON_EXISTING p50 latency in ms: 0.024499
GET_NON_EXISTING p90 latency in ms: 0.025753
GET_NON_EXISTING p99 latency in ms: 0.032353
GET_NON_EXISTING std dev in ms: 0.002189276292868018
Avg. time in ms per SET: 0.02604521729643955
SET p50 latency in ms: 0.025343
SET p90 latency in ms: 0.028023
SET p99 latency in ms: 0.034586
SET std dev in ms: 0.010441295957881177
Avg. time in ms per GET_EXISTING: 0.02502787202758837
GET_EXISTING p50 latency in ms: 0.024565
GET_EXISTING p90 latency in ms: 0.026185
GET_EXISTING p99 latency in ms: 0.032738
GET_EXISTING std dev in ms: 0.0024680822391800945

 =====> Jedis <===== 1 clients 100 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.025086793313259397
GET_NON_EXISTING p50 latency in ms: 0.024595
GET_NON_EXISTING p90 latency in ms: 0.026744
GET_NON_EXISTING p99 latency in ms: 0.032636
GET_NON_EXISTING std dev in ms: 0.005482908643326746
Avg. time in ms per SET: 0.026083205348481363
SET p50 latency in ms: 0.025394
SET p90 latency in ms: 0.028341
SET p99 latency in ms: 0.034495
SET std dev in ms: 0.006780192695191216
Avg. time in ms per GET_EXISTING: 0.02518680371507057
GET_EXISTING p50 latency in ms: 0.024652
GET_EXISTING p90 latency in ms: 0.026851
GET_EXISTING p99 latency in ms: 0.032708
GET_EXISTING std dev in ms: 0.01453752228129236

 =====> Jedis <===== 2 clients 100 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.024925301994765774
GET_NON_EXISTING p50 latency in ms: 0.024692
GET_NON_EXISTING p90 latency in ms: 0.025378
GET_NON_EXISTING p99 latency in ms: 0.03222
GET_NON_EXISTING std dev in ms: 0.007327866318762721
Avg. time in ms per SET: 0.02591111964605289
SET p50 latency in ms: 0.025515
SET p90 latency in ms: 0.026627
SET p99 latency in ms: 0.033651
SET std dev in ms: 0.01704989319400638
Avg. time in ms per GET_EXISTING: 0.025052059367382126
GET_EXISTING p50 latency in ms: 0.024777
GET_EXISTING p90 latency in ms: 0.025565
GET_EXISTING p99 latency in ms: 0.032552
GET_EXISTING std dev in ms: 0.009776029303772261


 =====> kotlin <===== 1 clients 10 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.09837112588984638
GET_NON_EXISTING p50 latency in ms: 0.077591
GET_NON_EXISTING p90 latency in ms: 0.088447
GET_NON_EXISTING p99 latency in ms: 0.161833
GET_NON_EXISTING std dev in ms: 2.2766930013114135
Avg. time in ms per SET: 2.773314145560116E-4
SET p50 latency in ms: 2.29E-4
SET p90 latency in ms: 3.69E-4
SET p99 latency in ms: 9.64E-4
SET std dev in ms: 7.31907655946929E-4
Avg. time in ms per GET_EXISTING: 0.09420931169541306
GET_EXISTING p50 latency in ms: 0.078092
GET_EXISTING p90 latency in ms: 0.088968
GET_EXISTING p99 latency in ms: 0.166218
GET_EXISTING std dev in ms: 1.807622756518303

 =====> kotlin <===== 2 clients 10 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.09074643403808111
GET_NON_EXISTING p50 latency in ms: 0.080023
GET_NON_EXISTING p90 latency in ms: 0.091632
GET_NON_EXISTING p99 latency in ms: 0.104237
GET_NON_EXISTING std dev in ms: 1.039947251415548
Avg. time in ms per SET: 2.931733077776657E-4
SET p50 latency in ms: 3.04E-4
SET p90 latency in ms: 3.78E-4
SET p99 latency in ms: 4.71E-4
SET std dev in ms: 3.070658997459155E-4
Avg. time in ms per GET_EXISTING: 0.09525075785180787
GET_EXISTING p50 latency in ms: 0.080535
GET_EXISTING p90 latency in ms: 0.092077
GET_EXISTING p99 latency in ms: 0.104938
GET_EXISTING std dev in ms: 1.3942664267218827

 =====> kotlin <===== 1 clients 100 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.09565358155741278
GET_NON_EXISTING p50 latency in ms: 0.076214
GET_NON_EXISTING p90 latency in ms: 0.084642
GET_NON_EXISTING p99 latency in ms: 0.09737
GET_NON_EXISTING std dev in ms: 1.840198665459457
Avg. time in ms per SET: 2.1858217840620058E-4
SET p50 latency in ms: 1.91E-4
SET p90 latency in ms: 3.12E-4
SET p99 latency in ms: 4.06E-4
SET std dev in ms: 1.8904026490950597E-4
Avg. time in ms per GET_EXISTING: 0.08956349393623905
GET_EXISTING p50 latency in ms: 0.076756
GET_EXISTING p90 latency in ms: 0.085276
GET_EXISTING p99 latency in ms: 0.097937
GET_EXISTING std dev in ms: 1.4253750474428886

 =====> kotlin <===== 2 clients 100 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.08564391204466068
GET_NON_EXISTING p50 latency in ms: 0.079537
GET_NON_EXISTING p90 latency in ms: 0.084765
GET_NON_EXISTING p99 latency in ms: 0.099163
GET_NON_EXISTING std dev in ms: 1.1280609370666068
Avg. time in ms per SET: 2.46170662054942E-4
SET p50 latency in ms: 2.29E-4
SET p90 latency in ms: 3.44E-4
SET p99 latency in ms: 4.28E-4
SET std dev in ms: 1.9071351952689995E-4
Avg. time in ms per GET_EXISTING: 0.09296368883717873
GET_EXISTING p50 latency in ms: 0.079799
GET_EXISTING p90 latency in ms: 0.085439
GET_EXISTING p99 latency in ms: 0.099398
GET_EXISTING std dev in ms: 1.6290809514535636

Babushka not yet configured

 =====> Lettuce <===== 1 clients 10 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.049090728618213876
GET_NON_EXISTING p50 latency in ms: 0.045486
GET_NON_EXISTING p90 latency in ms: 0.054234
GET_NON_EXISTING p99 latency in ms: 0.089939
GET_NON_EXISTING std dev in ms: 0.02082684347709462
Avg. time in ms per SET: 0.0522094731699985
SET p50 latency in ms: 0.048356
SET p90 latency in ms: 0.058725
SET p99 latency in ms: 0.105462
SET std dev in ms: 0.03528195703715755
Avg. time in ms per GET_EXISTING: 0.05003850161307919
GET_EXISTING p50 latency in ms: 0.046013
GET_EXISTING p90 latency in ms: 0.055201
GET_EXISTING p99 latency in ms: 0.094793
GET_EXISTING std dev in ms: 0.13535885837305997

 =====> Lettuce <===== 2 clients 10 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.04647130006207325
GET_NON_EXISTING p50 latency in ms: 0.045066
GET_NON_EXISTING p90 latency in ms: 0.050802
GET_NON_EXISTING p99 latency in ms: 0.065126
GET_NON_EXISTING std dev in ms: 0.004727763924072001
Avg. time in ms per SET: 0.04796566911876617
SET p50 latency in ms: 0.046416
SET p90 latency in ms: 0.052755
SET p99 latency in ms: 0.068721
SET std dev in ms: 0.005289249945344147
Avg. time in ms per GET_EXISTING: 0.046712003412331136
GET_EXISTING p50 latency in ms: 0.0453
GET_EXISTING p90 latency in ms: 0.051046
GET_EXISTING p99 latency in ms: 0.065883
GET_EXISTING std dev in ms: 0.005576487304503666

 =====> Lettuce <===== 1 clients 100 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.045279630549384345
GET_NON_EXISTING p50 latency in ms: 0.044405
GET_NON_EXISTING p90 latency in ms: 0.047746
GET_NON_EXISTING p99 latency in ms: 0.056058
GET_NON_EXISTING std dev in ms: 0.020610135928869452
Avg. time in ms per SET: 0.04648239029997591
SET p50 latency in ms: 0.04553
SET p90 latency in ms: 0.049323
SET p99 latency in ms: 0.058019
SET std dev in ms: 0.0033673329279624567
Avg. time in ms per GET_EXISTING: 0.04538805869431687
GET_EXISTING p50 latency in ms: 0.044592
GET_EXISTING p90 latency in ms: 0.047933
GET_EXISTING p99 latency in ms: 0.056506
GET_EXISTING std dev in ms: 0.003228286021000021

 =====> Lettuce <===== 2 clients 100 concurrent

Avg. time in ms per GET_NON_EXISTING: 0.045280652573839134
GET_NON_EXISTING p50 latency in ms: 0.044462
GET_NON_EXISTING p90 latency in ms: 0.047837
GET_NON_EXISTING p99 latency in ms: 0.056518
GET_NON_EXISTING std dev in ms: 0.004947987782067195
Avg. time in ms per SET: 0.046720531223178104
SET p50 latency in ms: 0.045718
SET p90 latency in ms: 0.049659
SET p99 latency in ms: 0.058472
SET std dev in ms: 0.0033873385358614847
Avg. time in ms per GET_EXISTING: 0.045471183672033
GET_EXISTING p50 latency in ms: 0.044675
GET_EXISTING p90 latency in ms: 0.04806
GET_EXISTING p99 latency in ms: 0.056693
GET_EXISTING std dev in ms: 0.0031597879029811237

@Yury-Fridlyand Yury-Fridlyand mentioned this pull request Oct 19, 2023
Closed
1 task
@Yury-Fridlyand Yury-Fridlyand mentioned this pull request Oct 26, 2023
Closed
Yury-Fridlyand added a commit that referenced this pull request Oct 26, 2023
@Yury-Fridlyand
Add uniffi Kotlin client from #25.
4ea66f2 
Signed-off-by: Yury-Fridlyand <[email protected]>
@Yury-Fridlyand Yury-Fridlyand mentioned this pull request Oct 27, 2023
Closed
@acarbonetto acarbonetto closed this Nov 8, 2023
@Yury-Fridlyand Yury-Fridlyand deleted the uniffi-kotlin-java-bench branch December 7, 2023 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jonathanl-bq jonathanl-bq Awaiting requested review from jonathanl-bq

@acarbonetto acarbonetto Awaiting requested review from acarbonetto

@Elen-Ghulam Elen-Ghulam Awaiting requested review from Elen-Ghulam

@SanHalacogluImproving SanHalacogluImproving Awaiting requested review from SanHalacogluImproving

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Yury-Fridlyand @acarbonetto