Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate more performant SavedObjects client/repository #194435

Open
ymao1 opened this issue Sep 30, 2024 · 2 comments
Open

Investigate more performant SavedObjects client/repository #194435

ymao1 opened this issue Sep 30, 2024 · 2 comments
Labels
discuss Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@ymao1
Copy link
Contributor

ymao1 commented Sep 30, 2024

In this PR: #193341, Response Ops is switching to perform partial updates on the rule saved object directly using an ES client instead of via the SavedObjectsClient. As part of the effort to scale the throughput of alerting rules to 10x, we identified several areas of performance bottleneck, one of which is using the SavedObjectsClient for the partial update because the SO client performs two ES requests for each update. This introduces some risk (outlined in the PR summary) because we're bypassing a lot of the built in audit logging, migration handling, encryption handling that the SavedObjects client provides. Response Ops decided this was a trade-off worth making for the performance gains provided by using the ES client directly to partially updated an allow-listed set of fields that are only updated during rule execution by the system user and are not included as AAD fields on the encrypted saved object.

From @rudolf:

At this point we're not aware of other performance use cases which need to avoid extra roundtrips at all costs.
I also think that Core should not try to solve every possible use case, but rather make it easy and as safe as possible for plugins to drop down one level of abstraction to enable unique requirements.
Perhaps ideally we'd expose something like the ApiExecutionContext so that you could e.g. still use encryption but not choose everything else

In this particular case, you actually probably don't want anything in saved objects, encryption, authorization, audit logging will all make the call slower
@ymao1 ymao1 added the Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc label Sep 30, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@TinaHeiligers
Copy link
Contributor

@ymao1 The SavedObjectsClient makes two ES requests for each update for BWC reasons. Updates also changed from partial to full because the doc gets updated client-side before the ES call and to ensure updates are allowed for each space the object exists in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

No branches or pull requests

3 participants