Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: prepare v0.10.0 release #2198

Merged
merged 1 commit into from
Oct 24, 2024
Merged

Conversation

mxinden
Copy link
Collaborator

@mxinden mxinden commented Oct 24, 2024

I would like to get #2184 into Firefox.

Early integration looks promising.

While previously we would spend significant CPU time in allocating and de-allocating received Datagrams:
Screenshot 2024-10-24 at 13-58-31 Firefox 133 – Linux – 10_24_2024 11 54 41 AM UTC (public) – Firefox Profiler

https://share.firefox.dev/40bq7Ka

None of this is present with v0.10.0:
Screenshot 2024-10-24 at 14-03-13 Firefox 133 – Linux – 10_24_2024 11 42 38 AM UTC (public) – Firefox Profiler

https://share.firefox.dev/3NCMBfv

@larseggert
Copy link
Collaborator

Did we want to switch to/enforce semver? (Not that this would be a point release.)

Copy link

codecov bot commented Oct 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.39%. Comparing base (b516fe9) to head (199f9a2).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2198      +/-   ##
==========================================
- Coverage   95.41%   95.39%   -0.03%     
==========================================
  Files         112      112              
  Lines       36447    36447              
==========================================
- Hits        34777    34767      -10     
- Misses       1670     1680      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mxinden
Copy link
Collaborator Author

mxinden commented Oct 24, 2024

Did we want to switch to/enforce semver? (Not that this would be a point release.)

Not sure I am following. Given that we are still < 1.0.0, we can make breaking changes in the minor version (i.e. xx.XX.xx). Once we go >1.0.0 we would need to bump the major version (i.e. XX.xx.xx).

See also https://semver.org/#how-should-i-deal-with-revisions-in-the-0yz-initial-development-phase.

Related: as discussed elsewhere, I would like to release v1.0.0 eventually. But that discussion doesn't need to start before, nor does it need to block this release.

Copy link

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

Benchmark results

Performance differences relative to b516fe9.

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [99.237 ns 99.580 ns 99.924 ns]
       change: [-0.1955% +0.3028% +0.8458%] (p = 0.26 > 0.05)

Found 10 outliers among 100 measurements (10.00%)
3 (3.00%) high mild
7 (7.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [117.70 ns 118.10 ns 118.53 ns]
       change: [-1.3111% +0.0444% +1.0187%] (p = 0.95 > 0.05)

Found 20 outliers among 100 measurements (20.00%)
2 (2.00%) low mild
1 (1.00%) high mild
17 (17.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [116.94 ns 117.29 ns 117.74 ns]
       change: [-0.0761% +0.3806% +0.8728%] (p = 0.13 > 0.05)

Found 16 outliers among 100 measurements (16.00%)
5 (5.00%) low severe
2 (2.00%) low mild
3 (3.00%) high mild
6 (6.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [96.875 ns 97.047 ns 97.244 ns]
       change: [-0.6560% +0.0548% +0.8507%] (p = 0.89 > 0.05)

Found 16 outliers among 100 measurements (16.00%)
8 (8.00%) high mild
8 (8.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [112.05 ms 112.10 ms 112.16 ms]
       change: [+0.6822% +0.7624% +0.8403%] (p = 0.00 < 0.05)

Found 9 outliers among 100 measurements (9.00%)
8 (8.00%) low mild
1 (1.00%) high mild

transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [25.918 ms 26.973 ms 28.030 ms]
       change: [-4.9890% +0.0909% +5.4159%] (p = 0.97 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [34.506 ms 36.223 ms 37.909 ms]
       change: [-9.8457% -3.2970% +3.4744%] (p = 0.35 > 0.05)
transfer/pacing-false/same-seed: No change in performance detected.
       time:   [25.179 ms 25.979 ms 26.788 ms]
       change: [-7.1916% -3.2319% +0.9496%] (p = 0.15 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-true/same-seed: No change in performance detected.
       time:   [39.875 ms 41.885 ms 43.942 ms]
       change: [-5.6432% +1.0900% +8.4766%] (p = 0.77 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

1-conn/1-100mb-resp/mtu-1500 (aka. Download)/client: Change within noise threshold.
       time:   [887.27 ms 895.81 ms 904.52 ms]
       thrpt:  [110.56 MiB/s 111.63 MiB/s 112.71 MiB/s]
change:
       time:   [+0.7852% +2.1318% +3.5230%] (p = 0.00 < 0.05)
       thrpt:  [-3.4031% -2.0873% -0.7790%]
1-conn/10_000-parallel-1b-resp/mtu-1500 (aka. RPS)/client: No change in performance detected.
       time:   [323.89 ms 327.07 ms 330.28 ms]
       thrpt:  [30.277 Kelem/s 30.575 Kelem/s 30.875 Kelem/s]
change:
       time:   [-0.0661% +1.4047% +2.9232%] (p = 0.07 > 0.05)
       thrpt:  [-2.8402% -1.3853% +0.0661%]
1-conn/1-1b-resp/mtu-1500 (aka. HPS)/client: No change in performance detected.
       time:   [33.611 ms 33.807 ms 34.022 ms]
       thrpt:  [29.393  elem/s 29.580  elem/s 29.752  elem/s]
change:
       time:   [-0.8845% -0.1597% +0.6428%] (p = 0.69 > 0.05)
       thrpt:  [-0.6387% +0.1599% +0.8924%]

Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe

1-conn/1-100mb-resp/mtu-65536 (aka. Download)/client: Change within noise threshold.
       time:   [112.01 ms 112.36 ms 112.68 ms]
       thrpt:  [887.46 MiB/s 889.98 MiB/s 892.78 MiB/s]
change:
       time:   [+0.8239% +1.2312% +1.6173%] (p = 0.00 < 0.05)
       thrpt:  [-1.5916% -1.2162% -0.8172%]

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low severe

1-conn/10_000-parallel-1b-resp/mtu-65536 (aka. RPS)/client: No change in performance detected.
       time:   [317.12 ms 320.91 ms 324.66 ms]
       thrpt:  [30.801 Kelem/s 31.161 Kelem/s 31.534 Kelem/s]
change:
       time:   [-0.0204% +1.5071% +3.1197%] (p = 0.06 > 0.05)
       thrpt:  [-3.0253% -1.4847% +0.0204%]
1-conn/1-1b-resp/mtu-65536 (aka. HPS)/client: Change within noise threshold.
       time:   [34.448 ms 34.662 ms 34.899 ms]
       thrpt:  [28.654  elem/s 28.850  elem/s 29.029  elem/s]
change:
       time:   [+0.8907% +1.7000% +2.5915%] (p = 0.00 < 0.05)
       thrpt:  [-2.5260% -1.6716% -0.8828%]

Found 14 outliers among 100 measurements (14.00%)
4 (4.00%) low mild
3 (3.00%) high mild
7 (7.00%) high severe

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing MTU Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 1504 197.9 ± 105.6 94.3 397.9 1.00
neqo msquic reno on 1504 214.4 ± 13.0 199.2 234.5 1.00
neqo msquic reno 1504 252.3 ± 79.3 210.9 500.4 1.00
neqo msquic cubic on 1504 286.0 ± 96.4 204.5 461.3 1.00
neqo msquic cubic 1504 219.6 ± 16.2 199.8 254.0 1.00
msquic neqo reno on 1504 681.5 ± 10.7 668.6 700.1 1.00
msquic neqo reno 1504 755.6 ± 73.4 695.8 913.5 1.00
msquic neqo cubic on 1504 697.5 ± 12.6 685.9 723.0 1.00
msquic neqo cubic 1504 731.2 ± 75.8 677.4 930.5 1.00
neqo neqo reno on 1504 423.0 ± 14.5 398.1 444.4 1.00
neqo neqo reno 1504 406.5 ± 13.3 389.7 426.2 1.00
neqo neqo cubic on 1504 458.8 ± 58.3 404.7 566.9 1.00
neqo neqo cubic 1504 422.1 ± 14.4 406.0 445.6 1.00
msquic msquic 65536 148.6 ± 90.9 91.7 380.4 1.00
neqo msquic reno on 65536 242.5 ± 71.7 198.4 441.5 1.00
neqo msquic reno 65536 210.4 ± 11.5 195.2 230.6 1.00
neqo msquic cubic on 65536 242.0 ± 69.7 209.6 460.7 1.00
neqo msquic cubic 65536 267.4 ± 104.1 199.9 467.4 1.00
msquic neqo reno on 65536 110.1 ± 65.8 82.9 349.4 1.00
msquic neqo reno 65536 106.1 ± 50.5 81.6 311.0 1.00
msquic neqo cubic on 65536 113.4 ± 56.8 80.4 328.8 1.00
msquic neqo cubic 65536 134.5 ± 78.4 83.2 354.8 1.00
neqo neqo reno on 65536 207.0 ± 130.3 123.3 574.0 1.00
neqo neqo reno 65536 201.8 ± 102.7 120.4 383.5 1.00
neqo neqo cubic on 65536 176.4 ± 106.5 123.8 469.9 1.00
neqo neqo cubic 65536 166.9 ± 63.5 119.1 380.3 1.00

⬇️ Download logs

@mxinden mxinden added this pull request to the merge queue Oct 24, 2024
Merged via the queue into mozilla:main with commit 214ad48 Oct 24, 2024
62 of 69 checks passed
@mxinden mxinden deleted the release-v0.10.0 branch October 24, 2024 16:38
@mxinden
Copy link
Collaborator Author

mxinden commented Oct 24, 2024

For the record:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants