Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] Optimized rule execution log performance #118925

Merged
merged 2 commits into from
Nov 24, 2021
Merged

[Security Solution] Optimized rule execution log performance #118925

merged 2 commits into from
Nov 24, 2021

Conversation

xcrzx
Copy link
Contributor

@xcrzx xcrzx commented Nov 17, 2021 •
edited
Loading

Summary

This PR addresses some of the performance issues listed here: #118511.

  • Removed extra write operation on initial rule status creation. Before we were writing an empty SO and then updating it, we are now writing SO already with attributes.
  • Removed extra write operation on updating rule execution status with metrics. Before we were updating execution status and then writing metrics, we are now writing status with metrics in a single operation.
  • Parallelized error status write operations.

Results

Total rule execution times for query rules went down ~30%, from 3 seconds on average to 2 seconds.

Before

Screenshot 2021-11-17 at 18 28 58
Screenshot 2021-11-17 at 18 29 07

After

Screenshot 2021-11-17 at 19 00 10
Screenshot 2021-11-17 at 19 00 22

@xcrzx xcrzx added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. auto-backport Deprecated - use backport:version if exact versions are needed v8.1.0 Team:Detection Rule Management Security Detection Rule Management Team labels Nov 17, 2021
@xcrzx xcrzx marked this pull request as ready for review November 17, 2021 19:18
@xcrzx xcrzx requested a review from a team as a code owner November 17, 2021 19:18
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@banderror banderror requested a review from a team November 18, 2021 12:11
@spong
Copy link
Member

spong commented Nov 24, 2021 •
edited
Loading

edit: Spoke w/ @xcrzx and the below behavior is expected and will be resolved as part of #119596

In testing I saw multiple going to run statuses being written at one point. Only the Endpoint Security rule was enabled and ran the endpoint data generator, then disabled and re-enabled the rule. I was not able to repro with subsequent data replays and enabling/disabling.

Discover_-_Elastic

spong
spong approved these changes Nov 24, 2021
View reviewed changes
Copy link
Member

@spong spong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked out, tested locally, verified execution logs were still being written as expected, and confirmed timing enhancements on the ops cluster. LGTM! 👍 🚀

Thanks for the optimizations here @xcrzx! 🙂

@spong
Copy link
Member

spong commented Nov 24, 2021

@elasticmachine merge upstream

@xcrzx xcrzx enabled auto-merge (squash) November 24, 2021 16:58
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @xcrzx

@xcrzx xcrzx merged commit d6ac415 into elastic:main Nov 24, 2021
@kibanamachine kibanamachine mentioned this pull request Nov 24, 2021
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Nov 24, 2021
@xcrzx @kibanamachine
Rule execution log performance optimizations (elastic#118925)
16eca33
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
✅ 8.0

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Nov 24, 2021
@kibanamachine @xcrzx
Rule execution log performance optimizations (#118925) (#119648)
16f5294 
Co-authored-by: Dmitry Shevchenko <[email protected]>
@xcrzx xcrzx mentioned this pull request Nov 25, 2021
Closed
4 tasks
TinLe pushed a commit to TinLe/kibana that referenced this pull request Dec 22, 2021
@xcrzx @TinLe
Rule execution log performance optimizations (elastic#118925)
5aec996
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@spong spong spong approved these changes

Assignees

@xcrzx xcrzx

Labels
auto-backport Deprecated - use backport:version if exact versions are needed performance release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.0.0 v8.1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@xcrzx @elasticmachine @spong @kibana-ci @kibanamachine @banderror