Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve coverage in modules #701

Open
IanHoang opened this issue Dec 4, 2024 · 0 comments
Open

Improve coverage in modules #701

IanHoang opened this issue Dec 4, 2024 · 0 comments
Labels
2.0.0 enhancement New feature or request good first issue Good for newcomers

Comments

@IanHoang
Copy link
Collaborator

IanHoang commented Dec 4, 2024

Is your feature request related to a problem? Please describe

When users run pytest --cov=. within the OSB repository, they get the following output.

---------- coverage: platform darwin, python 3.9.6-final-0 -----------
Name                                                                                          Stmts   Miss Branch BrPart  Cover
-------------------------------------------------------------------------------------------------------------------------------
benchmarks/__init__.py                                                                            0      0      0      0   100%
benchmarks/worker_coordinator/__init__.py                                                         0      0      0      0   100%
benchmarks/worker_coordinator/parsing_test.py                                                   113    113      0      0     0%
benchmarks/worker_coordinator/runner_test.py                                                     18     18      2      0     0%
benchmarks/workload/__init__.py                                                                   0      0      0      0   100%
benchmarks/workload/bulk_params_test.py                                                          62     62      0      0     0%
it/__init__.py                                                                                  149    149     26      0     0%
it/distribution_test.py                                                                          14     14      4      0     0%
it/download_test.py                                                                               8      8      2      0     0%
it/info_test.py                                                                                  17     17      0      0     0%
it/list_test.py                                                                                  18     18      0      0     0%
it/proxy_test.py                                                                                 50     50      2      0     0%
it/sources_test.py                                                                                8      8      0      0     0%
it/tracker_test.py                                                                               27     27      2      0     0%
osbenchmark/__init__.py                                                                          15      2      2      0    76%
osbenchmark/actor.py                                                                            120     90     36      0    19%
osbenchmark/aggregator.py                                                                       172     66     68     14    60%
osbenchmark/async_connection.py                                                                 140     67     34      4    49%
osbenchmark/benchmark.py                                                                        511    511    124      0     0%
osbenchmark/benchmarkd.py                                                                        62     62     24      0     0%
osbenchmark/builder/__init__.py                                                                   1      0      0      0   100%
osbenchmark/builder/builder.py                                                                  444    302    104      0    29%
osbenchmark/builder/cluster.py                                                                    9      1      0      0    89%
osbenchmark/builder/cluster_builder.py                                                           10     10      0      0     0%
osbenchmark/builder/configs/__init__.py                                                           0      0      0      0   100%
osbenchmark/builder/configs/listers/__init__.py                                                   0      0      0      0   100%
osbenchmark/builder/configs/listers/plugin_config_instance_lister.py                             40      0      6      0   100%
osbenchmark/builder/configs/utils/__init__.py                                                     0      0      0      0   100%
osbenchmark/builder/configs/utils/config_path_resolver.py                                        27      0      4      0   100%
osbenchmark/builder/downloaders/__init__.py                                                       0      0      0      0   100%
osbenchmark/builder/downloaders/builders/__init__.py                                              0      0      0      0   100%
osbenchmark/builder/downloaders/builders/binary_builder.py                                        5      1      0      0    80%
osbenchmark/builder/downloaders/builders/source_binary_builder.py                                27      0      2      0   100%
osbenchmark/builder/downloaders/core_plugin_source_downloader.py                                 19      0      2      1    95%
osbenchmark/builder/downloaders/downloader.py                                                     7      1      0      0    86%
osbenchmark/builder/downloaders/external_plugin_source_downloader.py                             23      0      2      1    96%
osbenchmark/builder/downloaders/opensearch_distribution_downloader.py                            45      3      2      0    94%
osbenchmark/builder/downloaders/opensearch_source_downloader.py                                  36      0      2      1    97%
osbenchmark/builder/downloaders/plugin_distribution_downloader.py                                 9      0      0      0   100%
osbenchmark/builder/downloaders/repositories/__init__.py                                          0      0      0      0   100%
osbenchmark/builder/downloaders/repositories/opensearch_distribution_repository_provider.py      22      0      2      0   100%
osbenchmark/builder/downloaders/repositories/plugin_distribution_repository_provider.py           8      0      0      0   100%
osbenchmark/builder/downloaders/repositories/repository_url_provider.py                          18      0      2      0   100%
osbenchmark/builder/downloaders/repositories/source_repository_provider.py                       64      0     12      1    99%
osbenchmark/builder/executors/__init__.py                                                         0      0      0      0   100%
osbenchmark/builder/executors/exception_handling_shell_executor.py                               10      0      0      0   100%
osbenchmark/builder/executors/local_shell_executor.py                                            10      0      4      0   100%
osbenchmark/builder/executors/shell_executor.py                                                   5      1      0      0    80%
osbenchmark/builder/installers/__init__.py                                                        0      0      0      0   100%
osbenchmark/builder/installers/bare_installer.py                                                 56      3     16      1    92%
osbenchmark/builder/installers/docker_installer.py                                               66     16      4      0    74%
osbenchmark/builder/installers/exception_handling_installer.py                                   16     16      0      0     0%
osbenchmark/builder/installers/installer.py                                                      10      2      0      0    80%
osbenchmark/builder/installers/preparers/__init__.py                                              0      0      0      0   100%
osbenchmark/builder/installers/preparers/opensearch_preparer.py                                  63      3      4      1    94%
osbenchmark/builder/installers/preparers/plugin_preparer.py                                      32      4      4      1    86%
osbenchmark/builder/installers/preparers/preparer.py                                             19      5      0      0    74%
osbenchmark/builder/java_resolver.py                                                             32      7     10      1    76%
osbenchmark/builder/launcher.py                                                                 185     12     42      7    92%
osbenchmark/builder/launchers/__init__.py                                                         0      0      0      0   100%
osbenchmark/builder/launchers/docker_launcher.py                                                 58      0      8      0   100%
osbenchmark/builder/launchers/exception_handling_launcher.py                                     16     16      0      0     0%
osbenchmark/builder/launchers/launcher.py                                                        10      2      0      0    80%
osbenchmark/builder/launchers/local_process_launcher.py                                         132     11     26      5    90%
osbenchmark/builder/launchers/no_op_launcher.py                                                   6      6      0      0     0%
osbenchmark/builder/models/__init__.py                                                            0      0      0      0   100%
osbenchmark/builder/models/architecture_types.py                                                 13      1      4      1    88%
osbenchmark/builder/models/bootstrap_phase.py                                                    12     12      4      0     0%
osbenchmark/builder/models/cluster.py                                                             7      7      0      0     0%
osbenchmark/builder/models/cluster_flavors.py                                                     4      0      0      0   100%
osbenchmark/builder/models/cluster_infra_providers.py                                             4      0      0      0   100%
osbenchmark/builder/models/config_instance_types.py                                               7      0      0      0   100%
osbenchmark/builder/models/host.py                                                                8      0      0      0   100%
osbenchmark/builder/models/node.py                                                               14      0      0      0   100%
osbenchmark/builder/models/plugin_config_instance.py                                             26      3      2      0    89%
osbenchmark/builder/models/provision_config_instance.py                                          30     30      2      0     0%
osbenchmark/builder/models/provision_config_instance_descriptor.py                               15     15      0      0     0%
osbenchmark/builder/models/provision_config_instance_types.py                                     4      4      0      0     0%
osbenchmark/builder/provision_config.py                                                         334     55    116      6    83%
osbenchmark/builder/provisioner.py                                                              304    100     60      9    65%
osbenchmark/builder/provisioners/__init__.py                                                      0      0      0      0   100%
osbenchmark/builder/provisioners/provisioner.py                                                  10     10      0      0     0%
osbenchmark/builder/supplier.py                                                                 515     83    128     15    82%
osbenchmark/builder/utils/__init__.py                                                             0      0      0      0   100%
osbenchmark/builder/utils/artifact_variables_provider.py                                         12      0      0      0   100%
osbenchmark/builder/utils/binary_keys.py                                                          3      0      0      0   100%
osbenchmark/builder/utils/config_applier.py                                                      32      0      8      0   100%
osbenchmark/builder/utils/git_manager.py                                                         16      7      0      0    56%
osbenchmark/builder/utils/host_cleaner.py                                                        14      0      4      0   100%
osbenchmark/builder/utils/java_home_resolver.py                                                  33      4      6      1    87%
osbenchmark/builder/utils/jdk_resolver.py                                                        54      1     22      3    95%
osbenchmark/builder/utils/path_manager.py                                                        20      0      4      1    96%
osbenchmark/builder/utils/template_renderer.py                                                   27      0      0      0   100%
osbenchmark/client.py                                                                           256     92     62     10    64%
osbenchmark/config.py                                                                           149     30     40      5    79%
osbenchmark/create_workload/__init__.py                                                           0      0      0      0   100%
osbenchmark/create_workload/config.py                                                            20      0      0      0   100%
osbenchmark/create_workload/extractors.py                                                       128     31     32      5    72%
osbenchmark/create_workload/helpers.py                                                          108    108     30      0     0%
osbenchmark/create_workload/workload_generator.py                                                50     50      4      0     0%
osbenchmark/exceptions.py                                                                        31      1      0      0    97%
osbenchmark/log.py                                                                               41     28     10      0    25%
osbenchmark/metrics.py                                                                         1149    159    360     65    83%
osbenchmark/paths.py                                                                             21      5      4      0    72%
osbenchmark/results_publisher.py                                                                418    296    122      7    25%
osbenchmark/telemetry.py                                                                       1050    148    308     36    83%
osbenchmark/test_execution_orchestrator.py                                                      223    142     30      1    35%
osbenchmark/time.py                                                                              52      4     12      5    86%
osbenchmark/utils/__init__.py                                                                     0      0      0      0   100%
osbenchmark/utils/collections.py                                                                 12      0     10      0   100%
osbenchmark/utils/console.py                                                                    117     21     20      6    80%
osbenchmark/utils/convert.py                                                                     46     16     14      4    63%
osbenchmark/utils/dataset.py                                                                    206     33     64     20    78%
osbenchmark/utils/git.py                                                                         76      7     18      4    88%
osbenchmark/utils/io.py                                                                         330    116     80      7    61%
osbenchmark/utils/jvm.py                                                                         61     14     24      2    76%
osbenchmark/utils/modules.py                                                                     55     38     20      0    23%
osbenchmark/utils/net.py                                                                        159    102     40      0    30%
osbenchmark/utils/opts.py                                                                       125     16     32      2    86%
osbenchmark/utils/parse.py                                                                       33     12     24      5    60%
osbenchmark/utils/periodic_waiter.py                                                             14      0      4      0   100%
osbenchmark/utils/process.py                                                                     86     30     16      1    64%
osbenchmark/utils/repo.py                                                                        93     27     32      1    70%
osbenchmark/utils/sysstats.py                                                                    39      6      6      3    80%
osbenchmark/utils/versions.py                                                                    95      5     40      3    94%
osbenchmark/version.py                                                                           29     10      8      2    57%
osbenchmark/worker_coordinator/__init__.py                                                        1      0      0      0   100%
osbenchmark/worker_coordinator/errors.py                                                         47     37     26      0    14%
osbenchmark/worker_coordinator/runner.py                                                       1632    182    470     46    86%
osbenchmark/worker_coordinator/scheduler.py                                                     121     13     18      3    88%
osbenchmark/worker_coordinator/worker_coordinator.py                                           1253    410    306     23    66%
osbenchmark/workload/__init__.py                                                                  2      0      0      0   100%
osbenchmark/workload/loader.py                                                                 1081    258    434     44    76%
osbenchmark/workload/params.py                                                                 1098     78    368     41    91%
osbenchmark/workload/workload.py                                                                658     87    232      5    86%
scripts/expand-data-corpus.py                                                                   119     67     28      5    39%
-------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                         15846   4715   4292    435    69%

Only 69% of the repository is covered with some modules lacking coverage.

Describe the solution you'd like

Add more unittests to improve coverage. This will help give us peace of mind when refactoring in the future or making changes for OSB 2.0.0 because we will know if we are introducing breaking changes. For areas with low coverage, we should aim to bring them up to at least 60-75% coverage.

It's important to note that high code coverage percentage does not guarantee high quality code coverage. However, low code coverage percentage informs us that areas of the code are untested. More importantly, we should ensure that the essential areas are properly covered by high quality tests.

@IanHoang IanHoang added enhancement New feature or request untriaged 2.0.0 good first issue Good for newcomers and removed untriaged labels Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.0.0 enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant