-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Security Solution][Serverless] Github tickets / notifications #197265
Conversation
Pinging @elastic/security-solution (Team: SecuritySolution) |
// Function to remove specific fields from an XML object in order to | ||
// compare them as strings. | ||
// eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
function removeFields(obj: any, fieldsToRemove: string[]): any { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
any reason for not having an explicit test for this function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Checking over the tests of the whole package junit_transformer
it looks like it is not expicitly tested as it relates to test code. I guess its not needed to try to keep a non existing test coverage. Also this part of code is tested by the generic test updates the file in place, applying the expected transformation
as all of the rest of the code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'd prefer an explicit test, but you sound confident enough in your response to me.
Thanks.
Oh, one last thing, I was not referring to coverage, but to my own personal nits about testing functions explicitly.
From your comment, it sounds like removeFields
is only tested implicitly.
As this lib.ts file is not actually owned by my team, no worries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CR only, LGTM
// Function to remove specific fields from an XML object in order to | ||
// compare them as strings. | ||
// eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
function removeFields(obj: any, fieldsToRemove: string[]): any { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'd prefer an explicit test, but you sound confident enough in your response to me.
Thanks.
Oh, one last thing, I was not referring to coverage, but to my own personal nits about testing functions explicitly.
From your comment, it sounds like removeFields
is only tested implicitly.
As this lib.ts file is not actually owned by my team, no worries.
function removeFields(obj: any, fieldsToRemove: string[]): any { | ||
for (const key in obj) { | ||
if (fieldsToRemove.includes(key)) { | ||
delete obj[key]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: I believe it would be greate if we can avoid the modification of the original object by making a copy first and doing the modifications there with the aim of avoid side effects.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But we do not change the actual file at any point. We just getting a parsed string out of it and then we remove the fields for future use.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prepend title 👍
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…#197265) (#199313) # Backport This will backport the following commits from `main` to `8.16`: - [[Security Solution][Serverless] Github tickets / notifications (#197265)](#197265) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"dkirchan","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-07T13:17:54Z","message":"[Security Solution][Serverless] Github tickets / notifications (#197265)\n\n## Summary\r\n\r\nThis PR is accompanied with the [PR in kibana-operations\r\nrepo](elastic/kibana-operations#236) and\r\nintegrates the security solution quality gate pipelines with the\r\nfunctionality to create ticket when a test fails and then notify the\r\nrespective **codeowner** team with a _Test failed alert_. Also when a\r\ntest fails a second time and a ticket exist a new comment is added in\r\nthe same issue that the test failed again in a given pipeline.\r\n\r\nLast a similar flow exist when a test is skipped by a team. \r\n\r\n**Specifications:**\r\n\r\n- In the `failed_test_reporter_cli.ts` [a new field is introduced\r\n](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle`\r\nwhich is initialized by the environmental variable\r\n`process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom\r\ntext of preference before the issue title.\r\n\r\nThe scope of this is to be able to give the team the opportunity to add\r\nsome tags or any convention agreed within the team in the issue created\r\nin order to easily understand the context without opening the ticket. If\r\nthis field is not initialized, the normal existing flow proceeds.\r\n[Here](#197133) an example of\r\nthe prependTitle usage can be seen where the tags `[MKI][QA]` are\r\nprepended in order to identify where did the test fail, having the same\r\nexactly tests running on both CI and MKI. This means that a github issue\r\nwith the exact same title would be created for both cases if this\r\nprepend title field would not exist.\r\n\r\n- In the\r\n[junit_transformer/lib.ts](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278)\r\na new functionality is introduced to cover an existing issue. The\r\nexisting issue is: the fact that we are retrying the failed test in\r\ncypress in the `parallel_serverless` script, leads to two junit xml\r\nresult files completely identical with only difference the execution\r\ntime and the timestamps. This change will take one by one the xml\r\noutputs, transform them exactly as it was happening before and then save\r\nthe file, but also remove the time and timestamp fields, convert it to a\r\nstring and add it to a \"state\" list. Then in a next file if it is\r\nalready parsed and saved to the list having the exact same results\r\nwithin, it deletes the file instead of saving it transformed.\r\n\r\nThe problem before this fix was the fact that when two xml outputs\r\nexisted, a github ticket was created and when parsing-uploading the\r\nsecond file it was immediately failing for a second time as well. This\r\nmeans that `new-failure` notification was never triggered after the\r\ngithub actions flow split, and the existing failure was always\r\ntriggered, something that most teams have disabled.\r\n\r\nWith the fix, the identical files are parsed but only once uploaded and\r\nthe new failure flow works again.","sha":"9353497d0adea457c13643bc8fa1a26a05422e8f","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","backport:prev-minor","v8.16.0","v8.17.0"],"title":"[Security Solution][Serverless] Github tickets / notifications","number":197265,"url":"https://github.com/elastic/kibana/pull/197265","mergeCommit":{"message":"[Security Solution][Serverless] Github tickets / notifications (#197265)\n\n## Summary\r\n\r\nThis PR is accompanied with the [PR in kibana-operations\r\nrepo](elastic/kibana-operations#236) and\r\nintegrates the security solution quality gate pipelines with the\r\nfunctionality to create ticket when a test fails and then notify the\r\nrespective **codeowner** team with a _Test failed alert_. Also when a\r\ntest fails a second time and a ticket exist a new comment is added in\r\nthe same issue that the test failed again in a given pipeline.\r\n\r\nLast a similar flow exist when a test is skipped by a team. \r\n\r\n**Specifications:**\r\n\r\n- In the `failed_test_reporter_cli.ts` [a new field is introduced\r\n](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle`\r\nwhich is initialized by the environmental variable\r\n`process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom\r\ntext of preference before the issue title.\r\n\r\nThe scope of this is to be able to give the team the opportunity to add\r\nsome tags or any convention agreed within the team in the issue created\r\nin order to easily understand the context without opening the ticket. If\r\nthis field is not initialized, the normal existing flow proceeds.\r\n[Here](#197133) an example of\r\nthe prependTitle usage can be seen where the tags `[MKI][QA]` are\r\nprepended in order to identify where did the test fail, having the same\r\nexactly tests running on both CI and MKI. This means that a github issue\r\nwith the exact same title would be created for both cases if this\r\nprepend title field would not exist.\r\n\r\n- In the\r\n[junit_transformer/lib.ts](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278)\r\na new functionality is introduced to cover an existing issue. The\r\nexisting issue is: the fact that we are retrying the failed test in\r\ncypress in the `parallel_serverless` script, leads to two junit xml\r\nresult files completely identical with only difference the execution\r\ntime and the timestamps. This change will take one by one the xml\r\noutputs, transform them exactly as it was happening before and then save\r\nthe file, but also remove the time and timestamp fields, convert it to a\r\nstring and add it to a \"state\" list. Then in a next file if it is\r\nalready parsed and saved to the list having the exact same results\r\nwithin, it deletes the file instead of saving it transformed.\r\n\r\nThe problem before this fix was the fact that when two xml outputs\r\nexisted, a github ticket was created and when parsing-uploading the\r\nsecond file it was immediately failing for a second time as well. This\r\nmeans that `new-failure` notification was never triggered after the\r\ngithub actions flow split, and the existing failure was always\r\ntriggered, something that most teams have disabled.\r\n\r\nWith the fix, the identical files are parsed but only once uploaded and\r\nthe new failure flow works again.","sha":"9353497d0adea457c13643bc8fa1a26a05422e8f"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197265","number":197265,"mergeCommit":{"message":"[Security Solution][Serverless] Github tickets / notifications (#197265)\n\n## Summary\r\n\r\nThis PR is accompanied with the [PR in kibana-operations\r\nrepo](elastic/kibana-operations#236) and\r\nintegrates the security solution quality gate pipelines with the\r\nfunctionality to create ticket when a test fails and then notify the\r\nrespective **codeowner** team with a _Test failed alert_. Also when a\r\ntest fails a second time and a ticket exist a new comment is added in\r\nthe same issue that the test failed again in a given pipeline.\r\n\r\nLast a similar flow exist when a test is skipped by a team. \r\n\r\n**Specifications:**\r\n\r\n- In the `failed_test_reporter_cli.ts` [a new field is introduced\r\n](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle`\r\nwhich is initialized by the environmental variable\r\n`process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom\r\ntext of preference before the issue title.\r\n\r\nThe scope of this is to be able to give the team the opportunity to add\r\nsome tags or any convention agreed within the team in the issue created\r\nin order to easily understand the context without opening the ticket. If\r\nthis field is not initialized, the normal existing flow proceeds.\r\n[Here](#197133) an example of\r\nthe prependTitle usage can be seen where the tags `[MKI][QA]` are\r\nprepended in order to identify where did the test fail, having the same\r\nexactly tests running on both CI and MKI. This means that a github issue\r\nwith the exact same title would be created for both cases if this\r\nprepend title field would not exist.\r\n\r\n- In the\r\n[junit_transformer/lib.ts](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278)\r\na new functionality is introduced to cover an existing issue. The\r\nexisting issue is: the fact that we are retrying the failed test in\r\ncypress in the `parallel_serverless` script, leads to two junit xml\r\nresult files completely identical with only difference the execution\r\ntime and the timestamps. This change will take one by one the xml\r\noutputs, transform them exactly as it was happening before and then save\r\nthe file, but also remove the time and timestamp fields, convert it to a\r\nstring and add it to a \"state\" list. Then in a next file if it is\r\nalready parsed and saved to the list having the exact same results\r\nwithin, it deletes the file instead of saving it transformed.\r\n\r\nThe problem before this fix was the fact that when two xml outputs\r\nexisted, a github ticket was created and when parsing-uploading the\r\nsecond file it was immediately failing for a second time as well. This\r\nmeans that `new-failure` notification was never triggered after the\r\ngithub actions flow split, and the existing failure was always\r\ntriggered, something that most teams have disabled.\r\n\r\nWith the fix, the identical files are parsed but only once uploaded and\r\nthe new failure flow works again.","sha":"9353497d0adea457c13643bc8fa1a26a05422e8f"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: dkirchan <[email protected]>
…ic#197265) ## Summary This PR is accompanied with the [PR in kibana-operations repo](elastic/kibana-operations#236) and integrates the security solution quality gate pipelines with the functionality to create ticket when a test fails and then notify the respective **codeowner** team with a _Test failed alert_. Also when a test fails a second time and a ticket exist a new comment is added in the same issue that the test failed again in a given pipeline. Last a similar flow exist when a test is skipped by a team. **Specifications:** - In the `failed_test_reporter_cli.ts` [a new field is introduced ](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle` which is initialized by the environmental variable `process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom text of preference before the issue title. The scope of this is to be able to give the team the opportunity to add some tags or any convention agreed within the team in the issue created in order to easily understand the context without opening the ticket. If this field is not initialized, the normal existing flow proceeds. [Here](elastic#197133) an example of the prependTitle usage can be seen where the tags `[MKI][QA]` are prepended in order to identify where did the test fail, having the same exactly tests running on both CI and MKI. This means that a github issue with the exact same title would be created for both cases if this prepend title field would not exist. - In the [junit_transformer/lib.ts](elastic/kibana@main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278) a new functionality is introduced to cover an existing issue. The existing issue is: the fact that we are retrying the failed test in cypress in the `parallel_serverless` script, leads to two junit xml result files completely identical with only difference the execution time and the timestamps. This change will take one by one the xml outputs, transform them exactly as it was happening before and then save the file, but also remove the time and timestamp fields, convert it to a string and add it to a "state" list. Then in a next file if it is already parsed and saved to the list having the exact same results within, it deletes the file instead of saving it transformed. The problem before this fix was the fact that when two xml outputs existed, a github ticket was created and when parsing-uploading the second file it was immediately failing for a second time as well. This means that `new-failure` notification was never triggered after the github actions flow split, and the existing failure was always triggered, something that most teams have disabled. With the fix, the identical files are parsed but only once uploaded and the new failure flow works again.
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
1 similar comment
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
This PR didn't make it on time to the latest 8.16.0 BC. Updating the labels. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
14 similar comments
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
…197265) (#199314) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution][Serverless] Github tickets / notifications (#197265)](#197265) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"dkirchan","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-07T13:17:54Z","message":"[Security Solution][Serverless] Github tickets / notifications (#197265)\n\n## Summary\r\n\r\nThis PR is accompanied with the [PR in kibana-operations\r\nrepo](elastic/kibana-operations#236) and\r\nintegrates the security solution quality gate pipelines with the\r\nfunctionality to create ticket when a test fails and then notify the\r\nrespective **codeowner** team with a _Test failed alert_. Also when a\r\ntest fails a second time and a ticket exist a new comment is added in\r\nthe same issue that the test failed again in a given pipeline.\r\n\r\nLast a similar flow exist when a test is skipped by a team. \r\n\r\n**Specifications:**\r\n\r\n- In the `failed_test_reporter_cli.ts` [a new field is introduced\r\n](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle`\r\nwhich is initialized by the environmental variable\r\n`process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom\r\ntext of preference before the issue title.\r\n\r\nThe scope of this is to be able to give the team the opportunity to add\r\nsome tags or any convention agreed within the team in the issue created\r\nin order to easily understand the context without opening the ticket. If\r\nthis field is not initialized, the normal existing flow proceeds.\r\n[Here](#197133) an example of\r\nthe prependTitle usage can be seen where the tags `[MKI][QA]` are\r\nprepended in order to identify where did the test fail, having the same\r\nexactly tests running on both CI and MKI. This means that a github issue\r\nwith the exact same title would be created for both cases if this\r\nprepend title field would not exist.\r\n\r\n- In the\r\n[junit_transformer/lib.ts](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278)\r\na new functionality is introduced to cover an existing issue. The\r\nexisting issue is: the fact that we are retrying the failed test in\r\ncypress in the `parallel_serverless` script, leads to two junit xml\r\nresult files completely identical with only difference the execution\r\ntime and the timestamps. This change will take one by one the xml\r\noutputs, transform them exactly as it was happening before and then save\r\nthe file, but also remove the time and timestamp fields, convert it to a\r\nstring and add it to a \"state\" list. Then in a next file if it is\r\nalready parsed and saved to the list having the exact same results\r\nwithin, it deletes the file instead of saving it transformed.\r\n\r\nThe problem before this fix was the fact that when two xml outputs\r\nexisted, a github ticket was created and when parsing-uploading the\r\nsecond file it was immediately failing for a second time as well. This\r\nmeans that `new-failure` notification was never triggered after the\r\ngithub actions flow split, and the existing failure was always\r\ntriggered, something that most teams have disabled.\r\n\r\nWith the fix, the identical files are parsed but only once uploaded and\r\nthe new failure flow works again.","sha":"9353497d0adea457c13643bc8fa1a26a05422e8f","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","backport:prev-minor","v8.16.0","v8.17.0"],"title":"[Security Solution][Serverless] Github tickets / notifications","number":197265,"url":"https://github.com/elastic/kibana/pull/197265","mergeCommit":{"message":"[Security Solution][Serverless] Github tickets / notifications (#197265)\n\n## Summary\r\n\r\nThis PR is accompanied with the [PR in kibana-operations\r\nrepo](elastic/kibana-operations#236) and\r\nintegrates the security solution quality gate pipelines with the\r\nfunctionality to create ticket when a test fails and then notify the\r\nrespective **codeowner** team with a _Test failed alert_. Also when a\r\ntest fails a second time and a ticket exist a new comment is added in\r\nthe same issue that the test failed again in a given pipeline.\r\n\r\nLast a similar flow exist when a test is skipped by a team. \r\n\r\n**Specifications:**\r\n\r\n- In the `failed_test_reporter_cli.ts` [a new field is introduced\r\n](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle`\r\nwhich is initialized by the environmental variable\r\n`process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom\r\ntext of preference before the issue title.\r\n\r\nThe scope of this is to be able to give the team the opportunity to add\r\nsome tags or any convention agreed within the team in the issue created\r\nin order to easily understand the context without opening the ticket. If\r\nthis field is not initialized, the normal existing flow proceeds.\r\n[Here](#197133) an example of\r\nthe prependTitle usage can be seen where the tags `[MKI][QA]` are\r\nprepended in order to identify where did the test fail, having the same\r\nexactly tests running on both CI and MKI. This means that a github issue\r\nwith the exact same title would be created for both cases if this\r\nprepend title field would not exist.\r\n\r\n- In the\r\n[junit_transformer/lib.ts](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278)\r\na new functionality is introduced to cover an existing issue. The\r\nexisting issue is: the fact that we are retrying the failed test in\r\ncypress in the `parallel_serverless` script, leads to two junit xml\r\nresult files completely identical with only difference the execution\r\ntime and the timestamps. This change will take one by one the xml\r\noutputs, transform them exactly as it was happening before and then save\r\nthe file, but also remove the time and timestamp fields, convert it to a\r\nstring and add it to a \"state\" list. Then in a next file if it is\r\nalready parsed and saved to the list having the exact same results\r\nwithin, it deletes the file instead of saving it transformed.\r\n\r\nThe problem before this fix was the fact that when two xml outputs\r\nexisted, a github ticket was created and when parsing-uploading the\r\nsecond file it was immediately failing for a second time as well. This\r\nmeans that `new-failure` notification was never triggered after the\r\ngithub actions flow split, and the existing failure was always\r\ntriggered, something that most teams have disabled.\r\n\r\nWith the fix, the identical files are parsed but only once uploaded and\r\nthe new failure flow works again.","sha":"9353497d0adea457c13643bc8fa1a26a05422e8f"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197265","number":197265,"mergeCommit":{"message":"[Security Solution][Serverless] Github tickets / notifications (#197265)\n\n## Summary\r\n\r\nThis PR is accompanied with the [PR in kibana-operations\r\nrepo](elastic/kibana-operations#236) and\r\nintegrates the security solution quality gate pipelines with the\r\nfunctionality to create ticket when a test fails and then notify the\r\nrespective **codeowner** team with a _Test failed alert_. Also when a\r\ntest fails a second time and a ticket exist a new comment is added in\r\nthe same issue that the test failed again in a given pipeline.\r\n\r\nLast a similar flow exist when a test is skipped by a team. \r\n\r\n**Specifications:**\r\n\r\n- In the `failed_test_reporter_cli.ts` [a new field is introduced\r\n](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-bb801c18fd2e1a3a36a3b39fbf02c1abe337c46c201ad5a01239a9d2501d4b56R47)`prependTitle`\r\nwhich is initialized by the environmental variable\r\n`process.env.PREPEND_FAILURE_TITLE` if exists, in order to add a custom\r\ntext of preference before the issue title.\r\n\r\nThe scope of this is to be able to give the team the opportunity to add\r\nsome tags or any convention agreed within the team in the issue created\r\nin order to easily understand the context without opening the ticket. If\r\nthis field is not initialized, the normal existing flow proceeds.\r\n[Here](#197133) an example of\r\nthe prependTitle usage can be seen where the tags `[MKI][QA]` are\r\nprepended in order to identify where did the test fail, having the same\r\nexactly tests running on both CI and MKI. This means that a github issue\r\nwith the exact same title would be created for both cases if this\r\nprepend title field would not exist.\r\n\r\n- In the\r\n[junit_transformer/lib.ts](https://github.com/elastic/kibana/compare/main...security-mki-github-tickets#diff-31c5651af613c7d02139f3e9fccd00ddb997f2502523372dd19db9e0659a66d6R278)\r\na new functionality is introduced to cover an existing issue. The\r\nexisting issue is: the fact that we are retrying the failed test in\r\ncypress in the `parallel_serverless` script, leads to two junit xml\r\nresult files completely identical with only difference the execution\r\ntime and the timestamps. This change will take one by one the xml\r\noutputs, transform them exactly as it was happening before and then save\r\nthe file, but also remove the time and timestamp fields, convert it to a\r\nstring and add it to a \"state\" list. Then in a next file if it is\r\nalready parsed and saved to the list having the exact same results\r\nwithin, it deletes the file instead of saving it transformed.\r\n\r\nThe problem before this fix was the fact that when two xml outputs\r\nexisted, a github ticket was created and when parsing-uploading the\r\nsecond file it was immediately failing for a second time as well. This\r\nmeans that `new-failure` notification was never triggered after the\r\ngithub actions flow split, and the existing failure was always\r\ntriggered, something that most teams have disabled.\r\n\r\nWith the fix, the identical files are parsed but only once uploaded and\r\nthe new failure flow works again.","sha":"9353497d0adea457c13643bc8fa1a26a05422e8f"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: dkirchan <[email protected]> Co-authored-by: Alex Szabo <[email protected]>
Summary
This PR is accompanied with the PR in kibana-operations repo and integrates the security solution quality gate pipelines with the functionality to create ticket when a test fails and then notify the respective codeowner team with a Test failed alert. Also when a test fails a second time and a ticket exist a new comment is added in the same issue that the test failed again in a given pipeline.
Last a similar flow exist when a test is skipped by a team.
Specifications:
failed_test_reporter_cli.ts
a new field is introducedprependTitle
which is initialized by the environmental variableprocess.env.PREPEND_FAILURE_TITLE
if exists, in order to add a custom text of preference before the issue title.The scope of this is to be able to give the team the opportunity to add some tags or any convention agreed within the team in the issue created in order to easily understand the context without opening the ticket. If this field is not initialized, the normal existing flow proceeds. Here an example of the prependTitle usage can be seen where the tags
[MKI][QA]
are prepended in order to identify where did the test fail, having the same exactly tests running on both CI and MKI. This means that a github issue with the exact same title would be created for both cases if this prepend title field would not exist.parallel_serverless
script, leads to two junit xml result files completely identical with only difference the execution time and the timestamps. This change will take one by one the xml outputs, transform them exactly as it was happening before and then save the file, but also remove the time and timestamp fields, convert it to a string and add it to a "state" list. Then in a next file if it is already parsed and saved to the list having the exact same results within, it deletes the file instead of saving it transformed.The problem before this fix was the fact that when two xml outputs existed, a github ticket was created and when parsing-uploading the second file it was immediately failing for a second time as well. This means that
new-failure
notification was never triggered after the github actions flow split, and the existing failure was always triggered, something that most teams have disabled.With the fix, the identical files are parsed but only once uploaded and the new failure flow works again.