Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(benchmark): avoid compiler optimizations in benchmarks loops. #2083

Merged
merged 1 commit into from
Sep 26, 2024

Conversation

FedeDP
Copy link
Contributor

@FedeDP FedeDP commented Sep 26, 2024

What type of PR is this?

/kind cleanup

Any specific area of the project related to this PR?

/area tests

Does this PR require a change in the driver versions?

What this PR does / why we need it:

See https://github.com/google/benchmark/blob/main/docs/user_guide.md#preventing-optimization.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

@poiana
Copy link
Contributor

poiana commented Sep 26, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@FedeDP
Copy link
Contributor Author

FedeDP commented Sep 26, 2024

/milestone 0.19.0

@poiana poiana added this to the 0.19.0 milestone Sep 26, 2024
@poiana poiana added the size/M label Sep 26, 2024
Copy link

Perf diff from master - unit tests

     5.11%     +1.05%  [.] next
     4.08%     -0.98%  [.] sinsp_thread_manager::get_thread_ref
     4.41%     -0.85%  [.] gzfile_read
     1.56%     +0.81%  [.] std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
     1.67%     -0.73%  [.] std::vector<sinsp_evt_param, std::allocator<sinsp_evt_param> >::emplace_back<sinsp_evt*, unsigned int&, char const*, unsigned long&>
     0.11%     +0.71%  [.] scap_get_event_info_table
     0.91%     +0.65%  [.] std::_Hashtable<long, std::pair<long const, std::shared_ptr<sinsp_threadinfo> >, std::allocator<std::pair<long const, std::shared_ptr<sinsp_threadinfo> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::find
     2.16%     +0.64%  [.] scap_event_decode_params
     1.24%     +0.63%  [.] libsinsp::events::is_unknown_event
     5.32%     +0.62%  [.] sinsp_evt::get_type

Heap diff from master - unit tests

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            -0.0015         -0.0015           149           149           149           149
BM_sinsp_split_median                                          +0.0013         +0.0013           149           150           149           150
BM_sinsp_split_stddev                                          +0.6234         +0.6231             1             1             1             1
BM_sinsp_split_cv                                              +0.6259         +0.6255             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  +0.3742         +0.3742            44            60            44            60
BM_sinsp_concatenate_paths_relative_path_median                +0.3623         +0.3623            44            60            44            60
BM_sinsp_concatenate_paths_relative_path_stddev                -0.8432         -0.8432             2             0             2             0
BM_sinsp_concatenate_paths_relative_path_cv                    -0.8859         -0.8859             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     +0.3506         +0.3506            18            24            18            24
BM_sinsp_concatenate_paths_empty_path_median                   +0.3462         +0.3461            18            24            18            24
BM_sinsp_concatenate_paths_empty_path_stddev                   -0.9144         -0.9141             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       -0.9366         -0.9364             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  +0.4066         +0.4066            45            63            45            63
BM_sinsp_concatenate_paths_absolute_path_median                +0.4084         +0.4084            45            63            45            63
BM_sinsp_concatenate_paths_absolute_path_stddev                +0.4045         +0.4047             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_cv                    -0.0015         -0.0013             0             0             0             0
BM_sinsp_split_container_image_mean                            +0.1366         +0.1366           350           398           350           398
BM_sinsp_split_container_image_median                          +0.1350         +0.1350           350           398           350           398
BM_sinsp_split_container_image_stddev                          +0.8627         +0.8578             1             2             1             2
BM_sinsp_split_container_image_cv                              +0.6388         +0.6345             0             0             0             0

Copy link

codecov bot commented Sep 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.58%. Comparing base (0eb7be3) to head (47bf180).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2083      +/-   ##
==========================================
- Coverage   73.58%   73.58%   -0.01%     
==========================================
  Files         253      253              
  Lines       31869    31869              
  Branches     5653     5623      -30     
==========================================
- Hits        23452    23451       -1     
+ Misses       8416     8387      -29     
- Partials        1       31      +30     
Flag Coverage Δ
libsinsp 73.58% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@poiana poiana merged commit 73bcad2 into master Sep 26, 2024
49 checks passed
@poiana poiana deleted the fix/do_not_let_compiler_optimize_benchmarks branch September 26, 2024 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants