Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[inference] add pre-bound versions of chatComplete and output APIs #200568

Merged
merged 16 commits into from
Nov 20, 2024
Merged

[inference] add pre-bound versions of chatComplete and output APIs #200568

merged 16 commits into from
Nov 20, 2024
+811 −61

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Nov 18, 2024 •
edited by kibanamachine
Loading

Summary

Fix #199084

Introduce pre-bound versions of the inference APIs.

Accessing the bound versions can be done using the same getClient API, via an additional bindTo parameter:

without bindings

const inferenceClient = myStartDeps.inference.getClient({ request });

const chatResponse = inferenceClient.chatComplete({
  connectorId: 'my-connector-id',
  functionCalling: 'simulated',
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});

with bindings

const inferenceClient = myStartDeps.inference.getClient({
  request,
  bindTo: {
   connectorId: 'my-connector-id',
   functionCalling: 'simulated',
  }
});

const chatResponse = inferenceClient.chatComplete({
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});

Note: this is only done for the server-side, as there isn't much value in scoping APIs on the browser side in my opinion

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet added release_note:skip Skip the PR/issue when compiling release notes Team:AI Infra AppEx AI Infrastructure Team v8.17.0 labels Nov 18, 2024
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet marked this pull request as ready for review November 19, 2024 07:32
@pgayvallet pgayvallet requested review from a team as code owners November 19, 2024 07:32
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-ai-infra (Team:AI Infra)

@pgayvallet pgayvallet added the backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) label Nov 19, 2024
legrego
legrego reviewed Nov 19, 2024
View reviewed changes
Copy link
Member

@legrego legrego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, I have a couple of comments for us to discuss. Great job with the docs and typings on this one too.

x-pack/plugins/inference/common/chat_complete/bind_chat_complete.ts Outdated Show resolved Hide resolved
x-pack/plugins/inference/server/inference_client/create_client.ts Show resolved Hide resolved
x-pack/plugins/inference/server/inference_client/inference_client.ts Show resolved Hide resolved
@pgayvallet pgayvallet changed the title [inference] add scoped versions of chatComplete and output APIs [inference] add pre-bound versions of chatComplete and output APIs Nov 20, 2024
@pgayvallet pgayvallet requested a review from legrego November 20, 2024 08:02
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
inference 23 25 +2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/inference-common 41 43 +2
inference 28 29 +1
total +3

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
inference 4 6 +2
Unknown metric groups

API count

id before after diff
@kbn/inference-common 124 132 +8
inference 33 40 +7
total +15

History

legrego
legrego approved these changes Nov 20, 2024
View reviewed changes
Copy link
Member

@legrego legrego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@pgayvallet
Copy link
Contributor Author

@elasticmachine merge upstream

@pgayvallet pgayvallet enabled auto-merge (squash) November 20, 2024 17:31
@pgayvallet pgayvallet merged commit 3c8f077 into elastic:main Nov 20, 2024
21 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11940107350

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Nov 20, 2024
@pgayvallet
[inference] add pre-bound versions of chatComplete and output APIs (
ee8a4c1 
elastic#200568)

## Summary

Fix elastic#199084

Introduce pre-bound versions of the inference APIs.

Accessing the bound versions can be done using the same `getClient` API,
via an additional `bindTo` parameter:

**without bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({ request });

const chatResponse = inferenceClient.chatComplete({
  connectorId: 'my-connector-id',
  functionCalling: 'simulated',
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

**with bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({
  request,
  bindTo: {
   connectorId: 'my-connector-id',
   functionCalling: 'simulated',
  }
});

const chatResponse = inferenceClient.chatComplete({
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

*Note: this is only done for the server-side, as there isn't much value
in scoping APIs on the browser side in my opinion*

---------

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 3c8f077)
@kibanamachine kibanamachine mentioned this pull request Nov 20, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Nov 20, 2024
@kibanamachine @pgayvallet
[8.x] [inference] add pre-bound versions of &#x60;chatComplete&#x60; …
29209cb 
…and &#x60;output&#x60; APIs (#200568) (#201028)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[inference] add pre-bound versions of &#x60;chatComplete&#x60; and
&#x60;output&#x60; APIs
(#200568)](#200568)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pierre
Gayvallet","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-20T19:09:11Z","message":"[inference]
add pre-bound versions of `chatComplete` and `output` APIs
(#200568)\n\n## Summary\r\n\r\nFix
https://github.com/elastic/kibana/issues/199084\r\n\r\nIntroduce
pre-bound versions of the inference APIs.\r\n\r\nAccessing the bound
versions can be done using the same `getClient` API,\r\nvia an
additional `bindTo` parameter:\r\n\r\n**without
bindings**\r\n```ts\r\nconst inferenceClient =
myStartDeps.inference.getClient({ request });\r\n\r\nconst chatResponse
= inferenceClient.chatComplete({\r\n connectorId: 'my-connector-id',\r\n
functionCalling: 'simulated',\r\n messages: [{ role: MessageRole.User,
content: 'Do something' }],\r\n});\r\n```\r\n\r\n**with
bindings**\r\n```ts\r\nconst inferenceClient =
myStartDeps.inference.getClient({\r\n request,\r\n bindTo: {\r\n
connectorId: 'my-connector-id',\r\n functionCalling: 'simulated',\r\n
}\r\n});\r\n\r\nconst chatResponse = inferenceClient.chatComplete({\r\n
messages: [{ role: MessageRole.User, content: 'Do something'
}],\r\n});\r\n```\r\n\r\n*Note: this is only done for the server-side,
as there isn't much value\r\nin scoping APIs on the browser side in my
opinion*\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"3c8f0777f4a4563824d0fb1f545524bf4346e3a2","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","Team:AI
Infra","v8.17.0"],"title":"[inference] add pre-bound versions of
`chatComplete` and `output`
APIs","number":200568,"url":"https://github.com/elastic/kibana/pull/200568","mergeCommit":{"message":"[inference]
add pre-bound versions of `chatComplete` and `output` APIs
(#200568)\n\n## Summary\r\n\r\nFix
https://github.com/elastic/kibana/issues/199084\r\n\r\nIntroduce
pre-bound versions of the inference APIs.\r\n\r\nAccessing the bound
versions can be done using the same `getClient` API,\r\nvia an
additional `bindTo` parameter:\r\n\r\n**without
bindings**\r\n```ts\r\nconst inferenceClient =
myStartDeps.inference.getClient({ request });\r\n\r\nconst chatResponse
= inferenceClient.chatComplete({\r\n connectorId: 'my-connector-id',\r\n
functionCalling: 'simulated',\r\n messages: [{ role: MessageRole.User,
content: 'Do something' }],\r\n});\r\n```\r\n\r\n**with
bindings**\r\n```ts\r\nconst inferenceClient =
myStartDeps.inference.getClient({\r\n request,\r\n bindTo: {\r\n
connectorId: 'my-connector-id',\r\n functionCalling: 'simulated',\r\n
}\r\n});\r\n\r\nconst chatResponse = inferenceClient.chatComplete({\r\n
messages: [{ role: MessageRole.User, content: 'Do something'
}],\r\n});\r\n```\r\n\r\n*Note: this is only done for the server-side,
as there isn't much value\r\nin scoping APIs on the browser side in my
opinion*\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"3c8f0777f4a4563824d0fb1f545524bf4346e3a2"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/200568","number":200568,"mergeCommit":{"message":"[inference]
add pre-bound versions of `chatComplete` and `output` APIs
(#200568)\n\n## Summary\r\n\r\nFix
https://github.com/elastic/kibana/issues/199084\r\n\r\nIntroduce
pre-bound versions of the inference APIs.\r\n\r\nAccessing the bound
versions can be done using the same `getClient` API,\r\nvia an
additional `bindTo` parameter:\r\n\r\n**without
bindings**\r\n```ts\r\nconst inferenceClient =
myStartDeps.inference.getClient({ request });\r\n\r\nconst chatResponse
= inferenceClient.chatComplete({\r\n connectorId: 'my-connector-id',\r\n
functionCalling: 'simulated',\r\n messages: [{ role: MessageRole.User,
content: 'Do something' }],\r\n});\r\n```\r\n\r\n**with
bindings**\r\n```ts\r\nconst inferenceClient =
myStartDeps.inference.getClient({\r\n request,\r\n bindTo: {\r\n
connectorId: 'my-connector-id',\r\n functionCalling: 'simulated',\r\n
}\r\n});\r\n\r\nconst chatResponse = inferenceClient.chatComplete({\r\n
messages: [{ role: MessageRole.User, content: 'Do something'
}],\r\n});\r\n```\r\n\r\n*Note: this is only done for the server-side,
as there isn't much value\r\nin scoping APIs on the browser side in my
opinion*\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"3c8f0777f4a4563824d0fb1f545524bf4346e3a2"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Pierre Gayvallet <[email protected]>
TattdCodeMonkey pushed a commit to TattdCodeMonkey/kibana that referenced this pull request Nov 21, 2024
@pgayvallet @elasticmachine @TattdCodeMonkey
[inference] add pre-bound versions of chatComplete and output APIs (
e7aca74 
elastic#200568)

## Summary

Fix elastic#199084

Introduce pre-bound versions of the inference APIs.

Accessing the bound versions can be done using the same `getClient` API,
via an additional `bindTo` parameter:

**without bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({ request });

const chatResponse = inferenceClient.chatComplete({
  connectorId: 'my-connector-id',
  functionCalling: 'simulated',
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

**with bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({
  request,
  bindTo: {
   connectorId: 'my-connector-id',
   functionCalling: 'simulated',
  }
});

const chatResponse = inferenceClient.chatComplete({
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

*Note: this is only done for the server-side, as there isn't much value
in scoping APIs on the browser side in my opinion*

---------

Co-authored-by: Elastic Machine <[email protected]>
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Nov 26, 2024
@pgayvallet @elasticmachine @paulinashakirova
[inference] add pre-bound versions of chatComplete and output APIs (
1edc947 
elastic#200568)

## Summary

Fix elastic#199084

Introduce pre-bound versions of the inference APIs.

Accessing the bound versions can be done using the same `getClient` API,
via an additional `bindTo` parameter:

**without bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({ request });

const chatResponse = inferenceClient.chatComplete({
  connectorId: 'my-connector-id',
  functionCalling: 'simulated',
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

**with bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({
  request,
  bindTo: {
   connectorId: 'my-connector-id',
   functionCalling: 'simulated',
  }
});

const chatResponse = inferenceClient.chatComplete({
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

*Note: this is only done for the server-side, as there isn't much value
in scoping APIs on the browser side in my opinion*

---------

Co-authored-by: Elastic Machine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
@pgayvallet @elasticmachine @CAWilson94
[inference] add pre-bound versions of chatComplete and output APIs (
e42b23e 
elastic#200568)

## Summary

Fix elastic#199084

Introduce pre-bound versions of the inference APIs.

Accessing the bound versions can be done using the same `getClient` API,
via an additional `bindTo` parameter:

**without bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({ request });

const chatResponse = inferenceClient.chatComplete({
  connectorId: 'my-connector-id',
  functionCalling: 'simulated',
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

**with bindings**
```ts
const inferenceClient = myStartDeps.inference.getClient({
  request,
  bindTo: {
   connectorId: 'my-connector-id',
   functionCalling: 'simulated',
  }
});

const chatResponse = inferenceClient.chatComplete({
  messages: [{ role: MessageRole.User, content: 'Do something' }],
});
```

*Note: this is only done for the server-side, as there isn't much value
in scoping APIs on the browser side in my opinion*

---------

Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@legrego legrego legrego approved these changes

@dgieselaar dgieselaar dgieselaar approved these changes

Assignees
No one assigned
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team:AI Infra AppEx AI Infrastructure Team v8.17.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[inference] Binding APIs to common parameters
5 participants
@pgayvallet @elasticmachine @kibanamachine @dgieselaar @legrego