[Security solution] AI connector changes #170150
Conversation
stephmilovic
added
release_note:skip
|
Pinging @elastic/security-threat-hunting (Team:Threat Hunting) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
| @@ -114,6 +114,8 @@ export class OpenAIConnector extends SubActionConnector<Config, Secrets> { | |||
| method: 'post', | |||
| responseSchema: RunActionResponseSchema, | |||
| data: sanitizedBody, | |||
| // give up to 2 minutes for response | |||
| timeout: 120000, | |||
There was a problem hiding this comment.
Have you tried this in practice? The thing I'm worrying about is that task manager could potentially "kill" the task if it hit's IT's timeout limit - which I believe can be extended but will need to look up how ...
TM can't really kill a task, but will basically ignore it if it runs over it's time-limit, and would then generate some kind of warning / error. And potentially retry it, but I believe it doesn't today ...
There was a problem hiding this comment.
See this issue #166561
There was a problem hiding this comment.
Have you tried this in practice?
We already have this length timeout on bedrock. @jamesspi has been running into issues with OpenAI not having enough time, and has been using Bedrock because it allows the extra time
There was a problem hiding this comment.
Also see this issue! https://github.com/elastic/security-team/issues/7873
There was a problem hiding this comment.
I think the default task timeout is 5 minutes and I don't think we set a custom timeout for action tasks so this shouldn't be cancelled due to timeout
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
(cherry picked from commit 75586c8)
# Backport This will backport the following commits from `main` to `8.11`: - [[Security solution] AI connector changes (#170150)](#170150) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Steph Milovic","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-30T22:31:47Z","message":"[Security solution] AI connector changes (#170150)","sha":"75586c8233ff83ab0df403d13d5a674c216d0271","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Threat Hunting","Team: SecuritySolution","Team:Threat Hunting:Explore","v8.11.0","v8.12.0"],"number":170150,"url":"https://github.com/elastic/kibana/pull/170150","mergeCommit":{"message":"[Security solution] AI connector changes (#170150)","sha":"75586c8233ff83ab0df403d13d5a674c216d0271"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/170150","number":170150,"mergeCommit":{"message":"[Security solution] AI connector changes (#170150)","sha":"75586c8233ff83ab0df403d13d5a674c216d0271"}}]}] BACKPORT--> Co-authored-by: Steph Milovic <[email protected]>
Summary
Two quick 8.11 fixes for GenAI connectors:
runApisub-action of OpenAI connector, allowing for more time for the request to fulfill for non-streaming requests (Feature Request: Adjustable response timeout per GenAI connector #166561, https://github.com/elastic/security-team/issues/7873)service: bedrockargument to the aws signature for bedrock requests. Their documentation said they could infer this value from the url, but this has proven to be false. Prevents this error that was happening for some regions: