Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Response Ops][Task Manager] Adding background task to mark removed task types as unrecognized #199057

Merged
merged 5 commits into from
Nov 11, 2024

Conversation

ymao1
Copy link
Contributor

@ymao1 ymao1 commented Nov 5, 2024

Resolves #192686

Summary

Creates a background task to search for removed task types and mark them as unrecognized. Removes the current logic that does this during the task claim cycle for both task claim strategies.

@ymao1 ymao1 force-pushed the tm-unrecognized-tasks branch from 4791b9e to 5d27ab3 Compare November 6, 2024 16:07
@@ -95,6 +95,34 @@ export class SampleTaskManagerFixturePlugin
return res.ok({ body: {} });
}
);

router.post(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the functional tests that test this functionality load data from an ES archive. doing this overwrites the task manager index so we have to ensure this task is scheduled before scheduling it to run.

@ymao1 ymao1 changed the title Adding background task to mark removed task types as unrecognized [Response Ops][Task Manager] Adding background task to mark removed task types as unrecognized Nov 6, 2024
@ymao1 ymao1 self-assigned this Nov 6, 2024
@ymao1 ymao1 added release_note:skip Skip the PR/issue when compiling release notes Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v9.0.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) v8.17.0 labels Nov 6, 2024
@ymao1 ymao1 force-pushed the tm-unrecognized-tasks branch from f16a7c1 to 6e43d46 Compare November 6, 2024 17:58
@ymao1 ymao1 marked this pull request as ready for review November 6, 2024 19:46
@ymao1 ymao1 requested a review from a team as a code owner November 6, 2024 19:46
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@ymao1 ymao1 requested review from pmuellr and adcoelho November 6, 2024 19:46
Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'd like some changes:

  • moving the body of the try in the run() method of the task
  • change search to not return source since we don't need it

Everything else is just commentary ...

...

argggggghhhhh

I did this entire PR review in VS code, and it published my review comment, but none of the individual comments. FFS. Will do another review here, copying it over ...

Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welp, here are the two comments I wanted to leave in the previous review. Can't find the other comments I made, but they weren't important.

Lesson learned: vscode PR reviews are not ready for prime time, at least for commenting / submitting the review

return () => {
return {
async run() {
try {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to have the body of the try moved to a local function, to make this easier to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9bb0899

const esClient = elasticsearch.client.asInternalUser;

const result = await esClient.search<ConcreteTaskInstance>({
index: TASK_MANAGER_INDEX,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make sure this search doesn't return source, since we don't need it, and don't know how big it could be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! Updated in 9bb0899

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

cc @ymao1

Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for the changes!

@ymao1 ymao1 merged commit be949d6 into elastic:main Nov 11, 2024
38 checks passed
@ymao1 ymao1 deleted the tm-unrecognized-tasks branch November 11, 2024 20:17
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11785574437

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Nov 11, 2024
…ask types as `unrecognized` (elastic#199057)

Resolves elastic#192686

## Summary

Creates a background task to search for removed task types and mark them
as unrecognized. Removes the current logic that does this during the
task claim cycle for both task claim strategies.

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit be949d6)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

tkajtoch pushed a commit to tkajtoch/kibana that referenced this pull request Nov 12, 2024
…ask types as `unrecognized` (elastic#199057)

Resolves elastic#192686

## Summary

Creates a background task to search for removed task types and mark them
as unrecognized. Removes the current logic that does this during the
task claim cycle for both task claim strategies.

---------

Co-authored-by: kibanamachine <[email protected]>
kibanamachine added a commit that referenced this pull request Nov 13, 2024
…oved task types as &#x60;unrecognized&#x60; (#199057) (#199706)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Response Ops][Task Manager] Adding background task to mark removed
task types as &#x60;unrecognized&#x60;
(#199057)](#199057)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ying
Mao","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-11T20:17:46Z","message":"[Response
Ops][Task Manager] Adding background task to mark removed task types as
`unrecognized` (#199057)\n\nResolves
https://github.com/elastic/kibana/issues/192686\r\n\r\n##
Summary\r\n\r\nCreates a background task to search for removed task
types and mark them\r\nas unrecognized. Removes the current logic that
does this during the\r\ntask claim cycle for both task claim
strategies.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"be949d66e43ae24e9bce4a13a4613ad00e1dce9a","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","v8.17.0"],"title":"[Response
Ops][Task Manager] Adding background task to mark removed task types as
`unrecognized`","number":199057,"url":"https://github.com/elastic/kibana/pull/199057","mergeCommit":{"message":"[Response
Ops][Task Manager] Adding background task to mark removed task types as
`unrecognized` (#199057)\n\nResolves
https://github.com/elastic/kibana/issues/192686\r\n\r\n##
Summary\r\n\r\nCreates a background task to search for removed task
types and mark them\r\nas unrecognized. Removes the current logic that
does this during the\r\ntask claim cycle for both task claim
strategies.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"be949d66e43ae24e9bce4a13a4613ad00e1dce9a"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199057","number":199057,"mergeCommit":{"message":"[Response
Ops][Task Manager] Adding background task to mark removed task types as
`unrecognized` (#199057)\n\nResolves
https://github.com/elastic/kibana/issues/192686\r\n\r\n##
Summary\r\n\r\nCreates a background task to search for removed task
types and mark them\r\nas unrecognized. Removes the current logic that
does this during the\r\ntask claim cycle for both task claim
strategies.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"be949d66e43ae24e9bce4a13a4613ad00e1dce9a"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Ying Mao <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Nov 18, 2024
…ask types as `unrecognized` (elastic#199057)

Resolves elastic#192686

## Summary

Creates a background task to search for removed task types and mark them
as unrecognized. Removes the current logic that does this during the
task claim cycle for both task claim strategies.

---------

Co-authored-by: kibanamachine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ResponseOps][Task Manager] unrecognized tasks are not updated when mget task claimer is used
4 participants