Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet] make agent policy bumpRevision async or move it to task manager #198203

Closed
shahzad31 opened this issue Oct 29, 2024 · 2 comments · Fixed by #198658
Closed

[Fleet] make agent policy bumpRevision async or move it to task manager #198203

shahzad31 opened this issue Oct 29, 2024 · 2 comments · Fixed by #198658
Assignees
@shahzad31
Labels
Team:Fleet Team label for Observability Data Collection Fleet team

Comments

@shahzad31
Copy link
Contributor

Synthetics performs bulk operation like create let's say 100s of package policies in bulk, some of those operations are paginated. let's say if we have 1000 monitors and page of 250 means, 1000 monitors deployed on same agent policy will end up bumping revision 10 times and also each call will have to wait for bumpRevision , which take quite a time compared to other operations.

Propose solution

Make bumpRevision optionally async

Right now route waits for bumpRevision, we should make it optional not to wait for that operation

Move to task manage

Move bumpRevision to a task manager, this would probably also provide batching bumpRevision.
@shahzad31 shahzad31 added the Team:Fleet Team label for Observability Data Collection Fleet team label Oct 29, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@nchaulet
Copy link
Member

nchaulet commented Oct 30, 2024 •
edited
Loading

I think that new behaviour should be opt-in like a new parameter asyncBumpAgentPolicyRevision (default to false) so it’s not a breaking change for people relying on sync agent policy bump revision (we have some automated scale test that rely on that)

@shahzad31 shahzad31 self-assigned this Nov 1, 2024
@shahzad31 shahzad31 mentioned this issue Nov 1, 2024
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 5, 2024
@shahzad31
[Fleet/Synthetics] Async bumpRevision to improve performance !! (elas…
b3c4014 
…tic#198658)

## Summary

Async bumpRevision to improve performance

Fixes elastic#198203

Sample trace

<img width="1451" alt="image"
src="https://github.com/user-attachments/assets/cdec5fb6-78e5-409f-9b09-6d23d2c7c381">

<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/c04ee1c2-18ac-4edc-97c6-65732ca307da">

(cherry picked from commit a809ced)
shahzad31 added a commit to shahzad31/kibana that referenced this issue Nov 6, 2024
@shahzad31
[Fleet/Synthetics] Async bumpRevision to improve performance !! (elas…
f69fdcf 
…tic#198658)

## Summary

Async bumpRevision to improve performance

Fixes elastic#198203

Sample trace

<img width="1451" alt="image"
src="https://github.com/user-attachments/assets/cdec5fb6-78e5-409f-9b09-6d23d2c7c381">

<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/c04ee1c2-18ac-4edc-97c6-65732ca307da">

(cherry picked from commit a809ced)

# Conflicts:
#	x-pack/plugins/fleet/server/services/package_policy.ts
mgadewoll pushed a commit to mgadewoll/kibana that referenced this issue Nov 7, 2024
@shahzad31 @mgadewoll
[Fleet/Synthetics] Async bumpRevision to improve performance !! (elas…
aa8ed88 
…tic#198658)

## Summary

Async bumpRevision to improve performance

Fixes elastic#198203


Sample trace 


<img width="1451" alt="image"
src="https://github.com/user-attachments/assets/cdec5fb6-78e5-409f-9b09-6d23d2c7c381">


<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/c04ee1c2-18ac-4edc-97c6-65732ca307da">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shahzad31 shahzad31

Labels
Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Fleet/Synthetics] Async bumpRevision to improve performance !! shahzad31/kibana
3 participants
@nchaulet @shahzad31 @elasticmachine