Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLM tasks] Add product documentation retrieval task #194379

Merged
merged 113 commits into from
Nov 19, 2024

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Sep 30, 2024

Summary

Close #193473
Close #193474

This PR utilize the documentation packages that are build via the tool introduced by #193847, allowing to install them in Kibana and expose documentation retrieval as an LLM task that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the assistant's config screen, which will expose a new tool for the assistant, retrieve_documentation (only implemented for the o11y assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

General architecture

Screenshot 2024-10-17 at 09 22 32

What this PR does

Adds two plugin:

  • productDocBase: contains all the logic related to product documentation installation, status, and search. This is meant to be a "low level" components only responsible for this specific part.

  • llmTasks: an higher level plugin that will contain various LLM tasks to be used by assistants and genAI consumers. The intent is not to have a single place to put all llm tasks, but more to have a default place where we can introduce new tasks from. (fwiw, the nlToEsql task will probably be moved to that plugin).

  • Add a retrieve_documentation tool registration for the o11y assistant

  • Add a component on the o11y assistant configuration page to install the product doc

(wiring the feature to the o11y assistant was done for testing purposes mostly, any addition / changes / enhancement should be done by the owning team - either in this PR or as a follow-up)

What is NOT included in this PR:

  • Wire product base feature to the security assistant (should be done by the owning team as a follow-up)

    • installation
    • utilization as tool
  • FTR tests: this is somewhat blocked by the same things we need to figure out for https://github.com/elastic/kibana-team/issues/1271

Screenshots

Installation from o11y assistant configuration page

Screenshot 2024-10-17 at 09 41 24

Example of output

Without product documentation installed

Screenshot 2024-10-10 at 09 59 41

With product documentation installed

Screenshot 2024-10-10 at 09 55 38

@pgayvallet pgayvallet marked this pull request as ready for review September 30, 2024 11:18
@pgayvallet pgayvallet requested a review from a team as a code owner September 30, 2024 11:18
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-ai-infra (Team:AI Infra)

@pgayvallet pgayvallet added the backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) label Sep 30, 2024
Copy link
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kibana.jsonc LGTM

@pgayvallet pgayvallet requested a review from a team as a code owner October 1, 2024 13:14
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

packages/kbn-optimizer/limits.yml

@pgayvallet pgayvallet marked this pull request as draft October 3, 2024 13:26
@pgayvallet
Copy link
Contributor Author

Moving back to draft as I will use the PR for the installation logic

@legrego legrego added the ci:cloud-deploy Create or update a Cloud deployment label Nov 14, 2024
Copy link
Member

@legrego legrego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for addressing my previous feedback. Switching to use the task manager makes a lot of sense for our use case.

I have some non-blocking comments that we should address, either here or in a follow up.

Additionally, I'd like to test the install procedure in a memory constrained environment, such as a 1GB ESS instance.

docs/user/security/audit-logging.asciidoc Outdated Show resolved Hide resolved
[INSTALL_ALL_TASK_TYPE]: {
title: 'Install all product documentation artifacts',
timeout: '10m',
maxAttempts: 3,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check with response ops about the cost attribute for all of our tasks (https://github.com/pgayvallet/kibana/blob/1ddad2f6cd6b23333a6d2070e50d705ec79ab2fd/x-pack/plugins/task_manager/server/task.ts#L21-L25).

We know this is a long-running task which is capable of emitting event loop utilization warnings. Right now our cost options are tiny, normal, and extra large. I'm not sure if this qualifies as extra large or not, but maybe response ops can help us determine the appropriate cost to select.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I asked Mike about this

Copy link
Contributor

@mikecote mikecote Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra large means it needs a lot of CPU and memory resources compared to other typical background tasks (5x more than normal tasks). For example, indicator match rules in security solution.

Setting extra large indicates to task manager that it should consider these types of tasks as the equivalent of five normal background tasks, so it reduces overall task concurrency on the running node to accommodate these tasks. In general we recommend optimizing the task type for memory and CPU instead of setting extra large as it does affect Kibana throughput on a given node quite a bit when they run.

taskId: string;
timeout?: number;
interval?: number;
}): Promise<boolean> => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the return value doesn't appear to be used anywhere

wait: true,
});

return res.ok<PerformInstallResponse>({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will incorrectly report installed: true if the task fails.
Steps:

  1. Change the artifactRepositoryUrl to something that will not resolve
  2. Navigate to o11y ai assistant settings
  3. Click the Install button to invoke this endpoint
  4. Wait for the request to complete. It will eventually return a successful response after TM exhausts all retry attempts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah.. working with remote task execution is quite awkward. The issue I have is that once executed, the task instance is deleted, so we can't even retrieve the output status from it. But maybe in case of error the instance is preserved... idk

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I fixed it in b9145ef.

What I learned though, is that in case of task failure, the failure isn't stored anywhere (the task instance object still gets deleted at the end of the run), so

  1. I have to manually detect if the task was successful or not (by checking install status after the run)...
  2. meaning I can't surface the failure cause to the UI...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pgayvallet nice, I think that'll work well enough given our constraints. Thanks for digging into this.

@pgayvallet
Copy link
Contributor Author

pgayvallet commented Nov 15, 2024

Additionally, I'd like to test the install procedure in a memory constrained environment, such as a 1GB ESS instance.

@legrego on serverless oblt project (with, AFAIK, 1gb BG nodes):

Screenshot 2024-11-15 at 13 58 36

Takes ~6mins, but install is successful

@pgayvallet
Copy link
Contributor Author

@elasticmachine merge upstream

@pgayvallet pgayvallet enabled auto-merge (squash) November 19, 2024 12:40
@elasticmachine
Copy link
Contributor

elasticmachine commented Nov 19, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
observabilityAiAssistantManagement 357 361 +4
productDocBase - 7 +7
total +11

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/product-doc-common - 31 +31
llmTasks - 1 +1
productDocBase - 10 +10
total +42

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
observabilityAiAssistantManagement 93.1KB 97.7KB +4.6KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
llmTasks - 2 +2
productDocBase - 4 +4
total +6

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
productDocBase - 2.2KB +2.2KB

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/saved-objects-service.html#_mappings

id before after diff
product-doc-install-status - 6 +6
Unknown metric groups

API count

id before after diff
@kbn/product-doc-common - 31 +31
llmTasks - 5 +5
productDocBase - 10 +10
total +46

ESLint disabled in files

id before after diff
llmTasks - 1 +1
productDocBase - 2 +2
total +3

Total ESLint disabled count

id before after diff
llmTasks - 1 +1
productDocBase - 2 +2
total +3

History

@pgayvallet pgayvallet merged commit 455c781 into elastic:main Nov 19, 2024
41 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11915163871

@pgayvallet
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

pgayvallet added a commit to pgayvallet/kibana that referenced this pull request Nov 19, 2024
## Summary

Close elastic#193473
Close elastic#193474

This PR utilize the documentation packages that are build via the tool
introduced by elastic#193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Alex Szabo <[email protected]>
Co-authored-by: Matthias Wilhelm <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 455c781)

# Conflicts:
#	.github/CODEOWNERS
pgayvallet added a commit that referenced this pull request Nov 19, 2024
…200754)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[LLM tasks] Add product documentation retrieval task
(#194379)](#194379)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pierre
Gayvallet","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-19T14:28:26Z","message":"[LLM
tasks] Add product documentation retrieval task (#194379)\n\n##
Summary\r\n\r\nClose
https://github.com/elastic/kibana/issues/193473\r\nClose
https://github.com/elastic/kibana/issues/193474\r\n\r\nThis PR utilize
the documentation packages that are build via the tool\r\nintroduced by
#193847, allowing to\r\ninstall
them in Kibana and expose documentation retrieval as an LLM task\r\nthat
AI assistants (or other consumers) can call.\r\n\r\nUsers can now decide
to install the Elastic documentation from the\r\nassistant's config
screen, which will expose a new tool for the\r\nassistant,
`retrieve_documentation` (only implemented for the o11y\r\nassistant in
the current PR, shall be done for security as a follow up).\r\n\r\nFor
more information, please refer to the self-review.\r\n\r\n## General
architecture\r\n\r\n<img width=\"1118\" alt=\"Screenshot 2024-10-17 at
09 22
32\"\r\nsrc=\"https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4\">\r\n\r\n##
What this PR does\r\n\r\nAdds two plugin:\r\n- `productDocBase`:
contains all the logic related to product\r\ndocumentation installation,
status, and search. This is meant to be a\r\n\"low level\" components
only responsible for this specific part.\r\n- `llmTasks`: an higher
level plugin that will contain various LLM tasks\r\nto be used by
assistants and genAI consumers. The intent is not to have\r\na single
place to put all llm tasks, but more to have a default place\r\nwhere we
can introduce new tasks from. (fwiw, the `nlToEsql` task
will\r\nprobably be moved to that plugin).\r\n\r\n- Add a
`retrieve_documentation` tool registration for the
o11y\r\nassistant\r\n- Add a component on the o11y assistant
configuration page to install\r\nthe product doc\r\n\r\n(wiring the
feature to the o11y assistant was done for testing purposes\r\nmostly,
any addition / changes / enhancement should be done by the\r\nowning
team - either in this PR or as a follow-up)\r\n\r\n## What is NOT
included in this PR:\r\n\r\n- Wire product base feature to the security
assistant (should be done by\r\nthe owning team as a follow-up)\r\n -
installation\r\n - utilization as tool\r\n\r\n- FTR tests: this is
somewhat blocked by the same things we need to\r\nfigure out for
https://github.com/elastic/kibana-team/issues/1271\r\n\r\n## Screenshots
\r\n\r\n### Installation from o11y assistant configuration
page\r\n\r\n<img width=\"1476\" alt=\"Screenshot 2024-10-17 at 09 41
24\"\r\nsrc=\"https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a\">\r\n\r\n###
Example of output\r\n\r\n#### Without product documentation installed
\r\n\r\n<img width=\"739\" alt=\"Screenshot 2024-10-10 at 09 59
41\"\r\nsrc=\"https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d\">\r\n\r\n####
With product documentation installed\r\n\r\n<img width=\"718\"
alt=\"Screenshot 2024-10-10 at 09 55
38\"\r\nsrc=\"https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Alex Szabo <[email protected]>\r\nCo-authored-by: Matthias Wilhelm
<[email protected]>\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"455c781c6d1e1161f66e275299cf06064a0ffde2","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","ci:cloud-deploy","Team:Obs
AI Assistant","ci:project-deploy-observability","Team:AI
Infra","v8.17.0"],"number":194379,"url":"https://github.com/elastic/kibana/pull/194379","mergeCommit":{"message":"[LLM
tasks] Add product documentation retrieval task (#194379)\n\n##
Summary\r\n\r\nClose
https://github.com/elastic/kibana/issues/193473\r\nClose
https://github.com/elastic/kibana/issues/193474\r\n\r\nThis PR utilize
the documentation packages that are build via the tool\r\nintroduced by
#193847, allowing to\r\ninstall
them in Kibana and expose documentation retrieval as an LLM task\r\nthat
AI assistants (or other consumers) can call.\r\n\r\nUsers can now decide
to install the Elastic documentation from the\r\nassistant's config
screen, which will expose a new tool for the\r\nassistant,
`retrieve_documentation` (only implemented for the o11y\r\nassistant in
the current PR, shall be done for security as a follow up).\r\n\r\nFor
more information, please refer to the self-review.\r\n\r\n## General
architecture\r\n\r\n<img width=\"1118\" alt=\"Screenshot 2024-10-17 at
09 22
32\"\r\nsrc=\"https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4\">\r\n\r\n##
What this PR does\r\n\r\nAdds two plugin:\r\n- `productDocBase`:
contains all the logic related to product\r\ndocumentation installation,
status, and search. This is meant to be a\r\n\"low level\" components
only responsible for this specific part.\r\n- `llmTasks`: an higher
level plugin that will contain various LLM tasks\r\nto be used by
assistants and genAI consumers. The intent is not to have\r\na single
place to put all llm tasks, but more to have a default place\r\nwhere we
can introduce new tasks from. (fwiw, the `nlToEsql` task
will\r\nprobably be moved to that plugin).\r\n\r\n- Add a
`retrieve_documentation` tool registration for the
o11y\r\nassistant\r\n- Add a component on the o11y assistant
configuration page to install\r\nthe product doc\r\n\r\n(wiring the
feature to the o11y assistant was done for testing purposes\r\nmostly,
any addition / changes / enhancement should be done by the\r\nowning
team - either in this PR or as a follow-up)\r\n\r\n## What is NOT
included in this PR:\r\n\r\n- Wire product base feature to the security
assistant (should be done by\r\nthe owning team as a follow-up)\r\n -
installation\r\n - utilization as tool\r\n\r\n- FTR tests: this is
somewhat blocked by the same things we need to\r\nfigure out for
https://github.com/elastic/kibana-team/issues/1271\r\n\r\n## Screenshots
\r\n\r\n### Installation from o11y assistant configuration
page\r\n\r\n<img width=\"1476\" alt=\"Screenshot 2024-10-17 at 09 41
24\"\r\nsrc=\"https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a\">\r\n\r\n###
Example of output\r\n\r\n#### Without product documentation installed
\r\n\r\n<img width=\"739\" alt=\"Screenshot 2024-10-10 at 09 59
41\"\r\nsrc=\"https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d\">\r\n\r\n####
With product documentation installed\r\n\r\n<img width=\"718\"
alt=\"Screenshot 2024-10-10 at 09 55
38\"\r\nsrc=\"https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Alex Szabo <[email protected]>\r\nCo-authored-by: Matthias Wilhelm
<[email protected]>\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"455c781c6d1e1161f66e275299cf06064a0ffde2"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/194379","number":194379,"mergeCommit":{"message":"[LLM
tasks] Add product documentation retrieval task (#194379)\n\n##
Summary\r\n\r\nClose
https://github.com/elastic/kibana/issues/193473\r\nClose
https://github.com/elastic/kibana/issues/193474\r\n\r\nThis PR utilize
the documentation packages that are build via the tool\r\nintroduced by
#193847, allowing to\r\ninstall
them in Kibana and expose documentation retrieval as an LLM task\r\nthat
AI assistants (or other consumers) can call.\r\n\r\nUsers can now decide
to install the Elastic documentation from the\r\nassistant's config
screen, which will expose a new tool for the\r\nassistant,
`retrieve_documentation` (only implemented for the o11y\r\nassistant in
the current PR, shall be done for security as a follow up).\r\n\r\nFor
more information, please refer to the self-review.\r\n\r\n## General
architecture\r\n\r\n<img width=\"1118\" alt=\"Screenshot 2024-10-17 at
09 22
32\"\r\nsrc=\"https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4\">\r\n\r\n##
What this PR does\r\n\r\nAdds two plugin:\r\n- `productDocBase`:
contains all the logic related to product\r\ndocumentation installation,
status, and search. This is meant to be a\r\n\"low level\" components
only responsible for this specific part.\r\n- `llmTasks`: an higher
level plugin that will contain various LLM tasks\r\nto be used by
assistants and genAI consumers. The intent is not to have\r\na single
place to put all llm tasks, but more to have a default place\r\nwhere we
can introduce new tasks from. (fwiw, the `nlToEsql` task
will\r\nprobably be moved to that plugin).\r\n\r\n- Add a
`retrieve_documentation` tool registration for the
o11y\r\nassistant\r\n- Add a component on the o11y assistant
configuration page to install\r\nthe product doc\r\n\r\n(wiring the
feature to the o11y assistant was done for testing purposes\r\nmostly,
any addition / changes / enhancement should be done by the\r\nowning
team - either in this PR or as a follow-up)\r\n\r\n## What is NOT
included in this PR:\r\n\r\n- Wire product base feature to the security
assistant (should be done by\r\nthe owning team as a follow-up)\r\n -
installation\r\n - utilization as tool\r\n\r\n- FTR tests: this is
somewhat blocked by the same things we need to\r\nfigure out for
https://github.com/elastic/kibana-team/issues/1271\r\n\r\n## Screenshots
\r\n\r\n### Installation from o11y assistant configuration
page\r\n\r\n<img width=\"1476\" alt=\"Screenshot 2024-10-17 at 09 41
24\"\r\nsrc=\"https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a\">\r\n\r\n###
Example of output\r\n\r\n#### Without product documentation installed
\r\n\r\n<img width=\"739\" alt=\"Screenshot 2024-10-10 at 09 59
41\"\r\nsrc=\"https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d\">\r\n\r\n####
With product documentation installed\r\n\r\n<img width=\"718\"
alt=\"Screenshot 2024-10-10 at 09 55
38\"\r\nsrc=\"https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Alex Szabo <[email protected]>\r\nCo-authored-by: Matthias Wilhelm
<[email protected]>\r\nCo-authored-by: Elastic Machine
<[email protected]>","sha":"455c781c6d1e1161f66e275299cf06064a0ffde2"}},{"branch":"8.x","label":"v8.17.0","labelRegex":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Nov 26, 2024
## Summary

Close elastic#193473
Close elastic#193474

This PR utilize the documentation packages that are build via the tool
introduced by elastic#193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots 

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed 

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Alex Szabo <[email protected]>
Co-authored-by: Matthias Wilhelm <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
throw new Error(`Timeout waiting for ML model ${modelId} to be deployed`);
};

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little late but any reason why you opted for custom retry logic instead of p-retry? Same for waitUntilTaskCompleted

delanni pushed a commit that referenced this pull request Nov 29, 2024
## Summary

Follow-up of #194379.

Turns out, the prefix for kibana system indices is `.kibana_*`, so our
indices were not considered as kibana system indices and causing
warnings when created, such as

```
Elasticsearch deprecation: 299 Elasticsearch-6db572c986d7e114b8b46f1d6f4169bed06717c5 "index name [.kibana-ai-product-doc-kibana] starts with a dot '.', in the next major version, index names starting with a dot are reserved for hidden indices and system indices"
Origin:kibana
```

This PR addresses it, by changing the product doc index names to follow
our system index pattern.
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Nov 29, 2024
## Summary

Follow-up of elastic#194379.

Turns out, the prefix for kibana system indices is `.kibana_*`, so our
indices were not considered as kibana system indices and causing
warnings when created, such as

```
Elasticsearch deprecation: 299 Elasticsearch-6db572c986d7e114b8b46f1d6f4169bed06717c5 "index name [.kibana-ai-product-doc-kibana] starts with a dot '.', in the next major version, index names starting with a dot are reserved for hidden indices and system indices"
Origin:kibana
```

This PR addresses it, by changing the product doc index names to follow
our system index pattern.

(cherry picked from commit d5cf0a6)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Nov 29, 2024
## Summary

Follow-up of elastic#194379.

Turns out, the prefix for kibana system indices is `.kibana_*`, so our
indices were not considered as kibana system indices and causing
warnings when created, such as

```
Elasticsearch deprecation: 299 Elasticsearch-6db572c986d7e114b8b46f1d6f4169bed06717c5 "index name [.kibana-ai-product-doc-kibana] starts with a dot '.', in the next major version, index names starting with a dot are reserved for hidden indices and system indices"
Origin:kibana
```

This PR addresses it, by changing the product doc index names to follow
our system index pattern.

(cherry picked from commit d5cf0a6)
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
## Summary

Close elastic#193473
Close elastic#193474

This PR utilize the documentation packages that are build via the tool
introduced by elastic#193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.

Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).

For more information, please refer to the self-review.

## General architecture

<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">

## What this PR does

Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).

- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc

(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)

## What is NOT included in this PR:

- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
  - installation
  - utilization as tool

- FTR tests: this is somewhat blocked by the same things we need to
figure out for elastic/kibana-team#1271

## Screenshots 

### Installation from o11y assistant configuration page

<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">

### Example of output

#### Without product documentation installed 

<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">

#### With product documentation installed

<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Alex Szabo <[email protected]>
Co-authored-by: Matthias Wilhelm <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
## Summary

Follow-up of elastic#194379.

Turns out, the prefix for kibana system indices is `.kibana_*`, so our
indices were not considered as kibana system indices and causing
warnings when created, such as

```
Elasticsearch deprecation: 299 Elasticsearch-6db572c986d7e114b8b46f1d6f4169bed06717c5 "index name [.kibana-ai-product-doc-kibana] starts with a dot '.', in the next major version, index names starting with a dot are reserved for hidden indices and system indices"
Origin:kibana
```

This PR addresses it, by changing the product doc index names to follow
our system index pattern.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:AI Infra AppEx AI Infrastructure Team Team:Obs AI Assistant Observability AI Assistant v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[KB] expose documentation retrieval as LLM task [KB] Add documentation packaging workflow