Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark for roundtrip connections #235

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

domenkozar
Copy link
Collaborator

What's really surprising is that once you add more cores to the benchmarks, it becomes orders of magnitude slower.

With -N1:

benchmarking connections/100
time                 14.78 ms   (14.66 ms .. 14.86 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 15.11 ms   (14.99 ms .. 15.33 ms)
std dev              388.5 μs   (236.6 μs .. 611.0 μs)
                        
benchmarking connections/1000
time                 146.5 ms   (143.9 ms .. 149.7 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 150.5 ms   (148.9 ms .. 152.0 ms)
std dev              2.242 ms   (1.489 ms .. 3.410 ms)
variance introduced by outliers: 12% (moderately inflated)
                        
benchmarking connections/10000
time                 1.523 s    (1.500 s .. 1.567 s)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.541 s    (1.532 s .. 1.551 s)
std dev              10.84 ms   (3.937 ms .. 14.65 ms)
variance introduced by outliers: 19% (moderately inflated)

With -N2:

benchmarking connections/100
time                 15.90 ms   (8.682 ms .. 29.22 ms)
                     0.395 R²   (0.345 R² .. 0.999 R²)
mean                 9.890 ms   (8.624 ms .. 14.89 ms)
std dev              6.383 ms   (164.5 μs .. 13.09 ms)
variance introduced by outliers: 96% (severely inflated)
                        
benchmarking connections/1000
time                 86.76 ms   (83.65 ms .. 93.80 ms)
                     0.995 R²   (0.984 R² .. 1.000 R²)
mean                 88.68 ms   (87.43 ms .. 91.65 ms)
std dev              3.044 ms   (1.942 ms .. 4.282 ms)
                        
benchmarking connections/10000
time                 860.6 ms   (636.3 ms .. 1.077 s)
                     0.987 R²   (0.987 R² .. 1.000 R²)
mean                 931.1 ms   (890.3 ms .. 971.8 ms)
std dev              52.23 ms   (24.33 ms .. 69.72 ms)
variance introduced by outliers: 19% (moderately inflated)

With -N4:

Benchmark bench-connections: RUNNING...
benchmarking connections/100
time                 80.88 ms   (1.093 ms .. 159.9 ms)
                     0.263 R²   (0.004 R² .. 0.799 R²)
mean                 75.93 ms   (34.91 ms .. 128.1 ms)
std dev              78.69 ms   (56.97 ms .. 108.9 ms)
variance introduced by outliers: 91% (severely inflated)
                        
benchmarking connections/1000
time                 864.1 ms   (-959.9 ms .. 3.006 s)
                     0.580 R²   (0.000 R² .. 1.000 R²)
mean                 893.1 ms   (405.7 ms .. 1.064 s)
std dev              326.1 ms   (16.56 ms .. 394.2 ms)
variance introduced by outliers: 73% (severely inflated)

With -N8:

benchmarking connections/100
time                 1.331 s    (1.024 s .. 2.048 s)
                     0.966 R²   (0.947 R² .. 1.000 R²)
mean                 1.088 s    (1.024 s .. 1.216 s)
std dev              128.0 ms   (86.18 μs .. 148.1 ms)
variance introduced by outliers: 23% (moderately inflated)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant