[Security solution] Use BedrockRuntimeClient to interact with converse APIs
#201046
+454
−1,359
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stephmilovic
added
release_note:skip
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
patrykkopycinski
approved these changes
Nov 20, 2024
pmuellr
approved these changes
Nov 20, 2024
There was a problem hiding this comment.
ResponseOps changes LGTM, given the connector execution is not queued (run as alert actions).
As in a prior PR review, might want to change CODEOWNERS to mark the gen_ai* and get_token* files as owned by your team, as you won't need to wait for a largely rubber-stamped PR review from ResponseOps :-)
💔 Build Failed
Failed CI StepsTest Failures
Metrics [docs]Page load bundle
History |
stephmilovic
commented
Nov 21, 2024
Comment on lines
+453
to
+458
| if ('stream' in res) { | ||
| const resultStream = res.stream as SmithyMessageDecoderStream<unknown>; | ||
| // splits the stream in two, [stream = consumer, tokenStream = token tracking] | ||
| const [stream, tokenStream] = tee(resultStream); | ||
| return { ...res, stream, tokenStream }; | ||
| } |
There was a problem hiding this comment.
@patrykkopycinski I wrote my own tee function to split the iterator into two streams which can be independently read from at different speeds. 🎉
|
Starting backport for target branches: 8.17, 8.x |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Nov 21, 2024
…rse APIs (elastic#201046) (cherry picked from commit e92ef08)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Nov 21, 2024
…rse APIs (elastic#201046) (cherry picked from commit e92ef08)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Nov 21, 2024
…teract with converse APIs (#201046) (#201288) # Backport This will backport the following commits from `main` to `8.17`: - [[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)](#201046) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Steph Milovic","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-21T20:38:24Z","message":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)","sha":"e92ef08689e7821c5d9fc7d776f301b2ceead770","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","Team:Security Generative AI","backport:version","v8.17.0","v8.18.0"],"title":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs","number":201046,"url":"https://github.com/elastic/kibana/pull/201046","mergeCommit":{"message":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)","sha":"e92ef08689e7821c5d9fc7d776f301b2ceead770"}},"sourceBranch":"main","suggestedTargetBranches":["8.17","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201046","number":201046,"mergeCommit":{"message":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)","sha":"e92ef08689e7821c5d9fc7d776f301b2ceead770"}},{"branch":"8.17","label":"v8.17.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Steph Milovic <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Nov 21, 2024
…eract with converse APIs (#201046) (#201289) # Backport This will backport the following commits from `main` to `8.x`: - [[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)](#201046) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Steph Milovic","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-21T20:38:24Z","message":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)","sha":"e92ef08689e7821c5d9fc7d776f301b2ceead770","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","Team:Security Generative AI","backport:version","v8.17.0","v8.18.0"],"title":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs","number":201046,"url":"https://github.com/elastic/kibana/pull/201046","mergeCommit":{"message":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)","sha":"e92ef08689e7821c5d9fc7d776f301b2ceead770"}},"sourceBranch":"main","suggestedTargetBranches":["8.17","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201046","number":201046,"mergeCommit":{"message":"[Security solution] Use `BedrockRuntimeClient` to interact with converse APIs (#201046)","sha":"e92ef08689e7821c5d9fc7d776f301b2ceead770"}},{"branch":"8.17","label":"v8.17.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Steph Milovic <[email protected]>
This was referenced Nov 21, 2024
paulinashakirova
pushed a commit
to paulinashakirova/kibana
that referenced
this pull request
Nov 26, 2024
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Dec 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Amazon's documentation recommends using the converse API with @aws-sdk/client-bedrock-runtime. In my previous PR, I was able to implement using the APIs directly only by overriding the API request within
BedrockRuntimeClient(NodeHttpHandler) and letting the class handle the decoding. However, when I needed to decode the stream in the actions plugin for token tracking I ran into blockers.Instead of hitting the converse APIs directly with axios, it seems we should use the AWS SDK in order to rely on them to handle their own encoding. To accomplish this, I reverted my changes from the previous converse APIs PR and instead implement a sub action called
bedrockClientSendwhich invokes the send method ofBedrockRuntimeClientusingConverseCommandorConverseStreamCommand.Using this strategy, I was able to easily retrieve the final chunk from the stream to calculate usage tokens.
Schema validation
BedrockRuntimeClient.sendtakes a class as the sole parameter, which I was not able to define using config schema:So I did need to use
schema.any()for thecommandarg.