Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROF-10589] Add microbenchmark for GVL profiling sampling #3989

Merged
merged 1 commit into from
Oct 11, 2024

Conversation

ivoanjo
Copy link
Member

@ivoanjo ivoanjo commented Oct 10, 2024

What does this PR do?

This PR adds a new profiler microbenchmark that tests the performance of sampling as triggered from the GVL profiling feature.

Motivation:

I've created this benchmark as part of validating the performance of this feature.

Additional Notes:

Profiling the profiler with ddprof, it looks like right now the biggest performance bottleneck is the cost of stack sampling.

This can be validated easily -- changing the depth of the stack for the testing thread makes a big difference.

How to test the change?

Validate that this benchmark gets picked up by our usual harness to run microbenchmarks.

**What does this PR do?**

This PR adds a new profiler microbenchmark that tests the performance
of sampling as triggered from the GVL profiling feature.

**Motivation:**

I've created this benchmark as part of validating the performance of
this feature.

**Additional Notes:**

Profiling the profiler with [ddprof](https://github.com/datadog/ddprof),
it looks like right now the biggest performance bottleneck is the cost
of stack sampling.

This can be validated easily -- changing the depth of the stack
for the testing thread makes a big difference.

**How to test the change?**

Validate that this benchmark gets picked up by our usual harness to run
microbenchmarks.
@ivoanjo ivoanjo added dev/testing Involves testing processes (e.g. RSpec) profiling Involves Datadog profiling dev/internal Other internal work that does not need to be included in the changelog labels Oct 10, 2024
@ivoanjo ivoanjo requested review from a team as code owners October 10, 2024 14:57
if RUBY_VERSION < '3.2'
if VALIDATE_BENCHMARK_MODE
# To simplify things, we allow this benchmark to be run in VALIDATE_BENCHMARK_MODE even though it's a no-op
$stderr.puts "Skipping benchmark because it requires Ruby 3.2 or newer"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Suggested change
$stderr.puts "Skipping benchmark because it requires Ruby 3.2 or newer"
warn "Skipping benchmark because it requires Ruby 3.2 or newer"
Do not print on stderr directly (...read more)

In Ruby, it is a good practice to use warn instead of $stderr.puts for issuing warning messages. The warn method is specifically designed for this purpose and its use makes the intention of the code clearer.

The importance of this rule lies in the fact that warn and $stderr.puts behave differently in certain situations. For example, warn will prepend the filename and line number to the warning message, which can be very helpful for debugging. Moreover, warn respects the -W command-line option for setting warning levels, while $stderr.puts does not.

To avoid violating this rule, replace any instances of $stderr.puts with warn when you want to issue a warning. Remember that the purpose of $stderr.puts is to write to the standard error, not to issue warnings. If you want to write to the standard error for reasons other than issuing warnings, $stderr.puts is the appropriate method to use.

View in Datadog  Leave us feedback  Documentation

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.85%. Comparing base (58b17f0) to head (5585532).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3989      +/-   ##
==========================================
- Coverage   97.87%   97.85%   -0.02%     
==========================================
  Files        1314     1314              
  Lines       78652    78652              
  Branches     3909     3909              
==========================================
- Hits        76978    76966      -12     
- Misses       1674     1686      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pr-commenter
Copy link

pr-commenter bot commented Oct 10, 2024

Benchmarks

Benchmark execution time: 2024-10-10 15:32:52

Comparing candidate commit 5585532 in PR branch ivoanjo/prof-10589-gvl-profiling-microbenchmark with baseline commit 58b17f0 in branch master.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 23 metrics, 2 unstable metrics.

scenario:profiler - sample timeline=false

  • 🟥 throughput [-0.430op/s; -0.398op/s] or [-6.246%; -5.773%]

@ivoanjo ivoanjo merged commit 7808575 into master Oct 11, 2024
252 checks passed
@ivoanjo ivoanjo deleted the ivoanjo/prof-10589-gvl-profiling-microbenchmark branch October 11, 2024 08:27
@github-actions github-actions bot added this to the 2.4.0 milestone Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev/internal Other internal work that does not need to be included in the changelog dev/testing Involves testing processes (e.g. RSpec) profiling Involves Datadog profiling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants