Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: X-Pack Cloud Security Posture API Tests.x-pack/test/cloud_security_posture_api/routes/benchmarks·ts - GET /internal/cloud_security_posture/benchmarks Get Benchmark API Verify cspm benchmark score is updated when muting rules #179815

Closed
kibanamachine opened this issue Apr 2, 2024 · 5 comments · Fixed by #185055
Assignees
Labels
8.15 candidate automation failed-test A test failure on a tracked branch, potentially flaky-test Team:Cloud Security Cloud Security team related

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Apr 2, 2024

A test failed on a tracked branch

TypeError: Cannot read properties of undefined (reading 'metadata')
    at getMockFinding (benchmarks.ts:59:16)
    at Context.<anonymous> (benchmarks.ts:161:30)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Apr 2, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Apr 2, 2024
@kibanamachine kibanamachine added the Team:Cloud Security Cloud Security team related label Apr 2, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Apr 2, 2024
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Apr 12, 2024
@mistic
Copy link
Member

mistic commented Apr 12, 2024

Skipped.

main: 82fdf04

@animehart
Copy link
Contributor

So this flakiness is caused by one of the function in the FTR. This function has the chance to return undefined (it does this randomly, I can only reproduce this on Flaky Test Runner and not locally),
an easy way to go around this is by using mock benchmark rules instead of getting the fetching the benchmark rules from saved object (which can fail)

@animehart animehart linked a pull request Jun 13, 2024 that will close this issue
@maxcold
Copy link
Contributor

maxcold commented Jun 17, 2024

@animehart can you provide more context on what is the function you are talking about, how it was discovered that it can return undefined randomly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.15 candidate automation failed-test A test failure on a tracked branch, potentially flaky-test Team:Cloud Security Cloud Security team related
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants