[Bug]: Test report creation fails due to components missing from build manifest

[gaiksaya]

Describe the bug

See https://build.ci.opensearch.org/job/integ-test/8758/console

Traceback (most recent call last):
  File "/var/jenkins/workspace/integ-test/./src/run_test_report.py", line 34, in <module>
    sys.exit(main())
  File "/var/jenkins/workspace/integ-test/./src/run_test_report.py", line 26, in main
    test_report_data = test_report_runner.update_data()
  File "/var/jenkins/workspace/integ-test/src/report_workflow/test_report_runner.py", line 69, in update_data
    self.test_report_data["components"].append(self.component_entry(component.name))
  File "/var/jenkins/workspace/integ-test/src/report_workflow/test_report_runner.py", line 93, in component_entry
    component["repository"] = self.bundle_manifest.components[component_name].repository
KeyError: 'security-analytics'

Security-analytics failed to build for 2.17.2.
This build failure should not affect the test-report generation.

To reproduce

Run

Running ./src/run_test_report.py manifests/2.17.2/opensearch-2.17.2-test.yml --artifact-paths opensearch=https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.17.2/10371/linux/arm64/tar --test-run-id 8758 --test-type integ-test --base-path https://ci.opensearch.org/ci/dbc/integ-test/2.17.2/10371/linux/arm64/tar --release-candidate 0 

Expected behavior

Test report generation should not be affected for components missing from build manifest. It should either just skip or show as not available

Screenshots

If applicable, add screenshots to help explain your problem.

Host / Environment

No response

Additional context

No response

Relevant log output

No response

[gaiksaya]

https://github.com/opensearch-project/opensearch-build/blob/main/src/report_workflow/test_report_runner.py#L93
Seems to be the issue

[zelinh]

This issue seems valid. We are reporting all components from the test manifest unless user specify the components in arguments.

opensearch-build/src/report_workflow/test_report_runner.py

Line 62 in 5b16eea

for component in self.test_components.select(focus=self.args.components):

What we could do is have another conditional check to see if the component from the test manifest is present in the bundle manifest. If not, meaning component failed and not performing any tests, we would skip reporting for this component.

[zelinh]

We have previous considered this case https://github.com/opensearch-project/opensearch-build/tree/main/src/report_workflow#why-are-some-component-testing-results-missing-or-unavailable
Seems like the new addition https://github.com/opensearch-project/opensearch-build/blob/main/src/report_workflow/test_report_runner.py#L93 trying to add more information to the report manifest is generating the data directly from the bundle manifest.

I will take a look and see how we could fix it.

[gaiksaya]

Closing this issue via #5090