Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweaks to ad load time testing #1407

Merged
merged 6 commits into from
Jun 10, 2024
Merged

Tweaks to ad load time testing #1407

merged 6 commits into from
Jun 10, 2024

Conversation

emma-imber
Copy link
Contributor

@emma-imber emma-imber commented Jun 6, 2024

What does this change?

Adds 10 more articles to be tested for ad load speed. Also round the final figure we show for load time to avoid long recurring decimals being outputted.

Why?

Adding a few more articles should improve the reliability of the testing but it doesn't speed it up dramatically - it still runs in less than 3 minutes. Looping through the articles we already had slowed things down a lot more. Plus I think having a wider variety of articles let's us balance out differences between them to be more representative of what's on the site.

Reliability

I've run the check using 30 articles 12 times. These are the consented results I saw:

4474
4486
4488
4488
4494
4500
4502
4506
4570
4659
4698
5453

The median value is 4501 ms and the interquartile range is 126.5

There's one big outlier of 5453 ms, but otherwise the results weren't miles away from each other

Copy link

changeset-bot bot commented Jun 6, 2024

⚠️ No Changeset found

Latest commit: 927350a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

github-actions bot commented Jun 6, 2024

Ad load time test results

For consented, top-above-nav took on average 4486ms to load.
For consentless, top-above-nav took on average 3199ms to load.

Test conditions:

  • 5mbps download speed
  • 1.5mbps upload speed
  • 150ms latency

@emma-imber emma-imber marked this pull request as ready for review June 7, 2024 08:52
@emma-imber emma-imber requested a review from a team as a code owner June 7, 2024 08:52
@emma-imber emma-imber merged commit 61f26d2 into main Jun 10, 2024
11 checks passed
@emma-imber emma-imber deleted the ei/load-time-test-tweaks branch June 10, 2024 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants