Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI Assistant docs for 8.9 #3549

Merged
merged 17 commits into from
Jul 25, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assistant/images/add-alert-context.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/assistant.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/icon-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/quick-prompts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/system-prompt.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
105 changes: 68 additions & 37 deletions docs/assistant/security-assistant.asciidoc
Original file line number Diff line number Diff line change
@@ -1,67 +1,65 @@
[[security-assistant]]
[chapter]
= Security Assistant
= AI Assistant

:frontmatter-description: The Elastic Security Assistant is a generative AI open-code chat assistant.
:frontmatter-description: The Elastic AI Assistant is a generative AI open-code chat assistant.
:frontmatter-tags-products: [security]
:frontmatter-tags-content-type: [overview]
:frontmatter-tags-user-goals: [get-started]

The Elastic Security Assistant utilizes generative AI to bolster your cybersecurity operations team. It allows users to interact with {elastic-sec} for tasks such as alert investigation, incident response, and query generation or conversion using natural language and much more.
The Elastic AI Assistant utilizes generative AI to bolster your cybersecurity operations team. It allows users to interact with {elastic-sec} for tasks such as alert investigation, incident response, and query generation or conversion using natural language and much more.

A connector for OpenAI and Azure OpenAI Service powers the Security Assistant.
A connector for OpenAI or Azure OpenAI Service powers the AI Assistant.

[role="screenshot"]
image::images/assistant.gif[Animation of the Security Assistant chat window,90%]
image::images/assistant.gif[Animation of the AI Assistant chat window,90%]

[IMPORTANT]
====
This is an initial release of the Elastic Security Assistant. While designed to enhance your analysis with smart dialogues, its capabilities are still developing. Users should leverage it sensibly as the reliability of its responses might vary. Your insights, patience, and feedback help us calibrate this feature for optimal use. Always cross-verify any returned advice for accurate threat detection and response, insights, and query generation.

Also, the data you provide to the Security Assistant is _not_ anonymized, and is stored and processed by the third-party AI provider. This includes any data used in conversations for analysis or context, such as alert or event data, detection rule configurations, and queries. Therefore, be careful about sharing any confidential or sensitive details while using this feature.
This is an initial release of the Elastic AI Assistant, designed to enhance your analysis with smart dialogues. Its capabilities are still developing. Users should exercise caution as the quality of its responses might vary. Your insights and feedback will help us improve this feature. Always cross-verify AI-generated advice for accuracy.
====

.Requirements
[sidebar]
--
* The Elastic Security Assistant and Generative AI connector are available in {stack} version 8.8.1 and later.
* The Elastic AI Assistant and Generative AI connector are available in {stack} version 8.8.1 and later.

* This feature requires an https://www.elastic.co/pricing[Enterprise subscription].

* You must have an account with a third-party generative AI provider, which the Security Assistant uses to generate responses. Supported providers are OpenAI (`gpt-3.5-turbo` model) and Azure OpenAI Service (any model).
* You must have an account with a third-party generative AI provider, which the AI Assistant uses to generate responses. Supported providers are OpenAI and Azure OpenAI Service.
--

[discrete]
[[data-information]]
== Your data and the AI Assistant

Elastic does not store or examine prompts sent to or results generated by the AI Assistant, or use this data for model training. This includes anything you send the model, such as alert or event data, detection rule configurations, queries, and prompts. However, any data you provide to AI Assistant will be processed by the third-party provider that you chose when setting up the Generative AI connector as part of the assistant setup.
benironside marked this conversation as resolved.
Show resolved Hide resolved

Elastic does not control the third-party tools, and assumes no responsibility or liability for their content, operation, or use, nor for any loss or damage that may arise from your using such tools. Please exercise caution when using AI tools with personal, sensitive, or confidential information. Any data you submit may be used by the provider for AI training or other purposes. There is no guarantee that the provider will keep any information you provide secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.
benironside marked this conversation as resolved.
Show resolved Hide resolved

NOTE: Elastic can automatically anonymize event data that you provide to AI Assistant as context. To learn more, refer to <<configure-ai-assistant, Configure AI Assistant>>.
benironside marked this conversation as resolved.
Show resolved Hide resolved


[discrete]
[[set-up-ai-assistant]]
== Set up the Security Assistant
== Set up the AI Assistant

You must complete these steps before you can use the Security Assistant:
You must complete these steps before you can use the AI Assistant:

. Create an API key with your AI provider to authenticate requests from the Security Assistant. You'll use this in a later step. Refer to the provider's documentation for generating API keys:
. Create an API key with your AI provider to authenticate requests from the AI Assistant. You'll use this in the next step. Refer to the provider's documentation for generating API keys:
+
* https://platform.openai.com/docs/api-reference[OpenAI]
* https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference[Azure OpenAI Service]

. Add the following feature flag to {kib}'s configuration settings:
+
`xpack.securitySolution.enableExperimental: ['assistantEnabled']`
+
The configuration method depends on your deployment type:
+
* *Self-managed (on-premises) deployments*: Add the feature flag to the `kibana.yml` file, which is used to {kibana-ref}/settings.html[configure {kib}], then restart {kib}.
* *{ecloud} deployments*: Use the YAML editor in the {ecloud} console to add the feature flag to {cloud}/ec-manage-kibana-settings.html[{kib} user settings].

. Create a {kibana-ref}/gen-ai-action-type.html[Generative AI connector] using the AI provider's API key and URL to configure communication between {elastic-sec} and the provider. You can do this in {kib} from *Stack Management* -> *Connectors*, or from within the Security Assistant.
+
NOTE: The Generative AI connector type requires the `assistantEnabled` feature flag for use.
. Create a {kibana-ref}/gen-ai-action-type.html[Generative AI connector] using the AI provider's API key and URL to configure communication between {elastic-sec} and the provider. You can do this in {kib} from *Stack Management* -> *Connectors*, or from within the AI Assistant.
benironside marked this conversation as resolved.
Show resolved Hide resolved

[discrete]
[[start-chatting]]
== Start chatting

To open the Security Assistant, press *Cmd + ;* (or *Ctrl + ;* in Windows) from anywhere in the {security-app}. This opens the *Welcome* chat interface, where you can ask general questions about {elastic-sec}.
To open the AI Assistant, press *Cmd + ;* (or *Ctrl + ;* on Windows) from anywhere in the {security-app}. This opens the *Welcome* chat interface, where you can ask general questions about {elastic-sec}.

You can also chat with the Security Assistant from several areas in {elastic-sec}, and context-specific data and prompts will populate your conversation.
You can also chat with the AI Assistant from several particular pages in {elastic-sec} where you can easily send context-specific data and prompts to AI Assistant.
joepeeples marked this conversation as resolved.
Show resolved Hide resolved

* <<view-alert-details, Alert details>> or Event details flyout: Click *Chat* while viewing the details of an alert or event.
* <<rules-ui-management, Rules page>>: Select one or more rules, then click the magic wand icon (🪄✨) at the top of the page next to the *Rules* title.
Expand All @@ -72,29 +70,62 @@ NOTE: All chat history and custom quick prompts persist in local browser storage

[discrete]
[[interact-with-assistant]]
== Interact with the Security Assistant
== Interact with the AI Assistant

Use these features to adjust and act on your conversations with the Security Assistant:
Use these features to adjust and act on your conversations with the AI Assistant:

* Select a _system prompt_ at the beginning of a conversation to establish how detailed and technical you want the Security Assistant's answers to be.
* Select a _system prompt_ at the beginning of a conversation to establish how detailed and technical you want the AI Assistant's answers to be.
+
[role="screenshot"]
image::images/system-prompt.gif[The system prompt drop-down menu,90%]
+
System prompts provide context to the model, informing its response. To create a custom system prompt, open the system prompts dropdown menu and click *+ Add new system prompt...*.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find the + Add new system prompt... label, but I did see + Add quick prompt... in the BC5:
prompt

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
System prompts provide context to the model, informing its response. To create a custom system prompt, open the system prompts dropdown menu and click *+ Add new system prompt...*.
System prompts provide context to the model, informing its response. To create a custom system prompt, open the system prompts drop-down menu and click *+ Add new system prompt...*.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find the + Add new system prompt... label, but I did see + Add quick prompt... in the BC5

Add new system prompt is inside the system prompt drop-down. The instructions do say to "open the system prompts drop-down" to click this, but maybe it could be clearer? The animated GIF also shows this, but slightly cuts off the Add new system prompt option at the bottom.

+
NOTE: The system prompt is only configurable at the start of a conversation. To reconfigure it, clear the chat and start a new conversation.
benironside marked this conversation as resolved.
Show resolved Hide resolved

* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}. Available quick prompts vary based on context. You can also add custom quick prompts for questions you frequently ask the Security Assistant.
* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be helpful to let users know that they'll need to click the Submit message button after selecting a prompt. I know it's a little hand-holdy, but it could be a small detail that new users miss.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a little less explicitly hand-holdy, and includes other things the user can do:

Suggested change
* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}.
* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}. This adds a suggested prompt to the entry area, which you can edit before submitting.

+
[role="screenshot"]
image::images/quick-prompts.png[Quick prompts highlighted below a conversation,90%]
+
Quick prompt availability varies based on context — for example the Alert summarization quick prompt appears when you open the AI Assistant while viewing an alert. You can customize existing quick prompts and create new ones. To do so, click *+ Add Quick prompt...* or click the gear in the upper right of the chat window to open Quick Prompts settings.
benironside marked this conversation as resolved.
Show resolved Hide resolved

* Use these buttons to perform actions in the conversation history and prompt entry area:

** *Add note to timeline* (image:images/icon-add-note.png[Add note icon,16,16]): Create a note in Timeline using the selected text.
** *Add to existing case* (image:images/icon-add-to-case.png[Add to case icon,19,16]): Add a comment to an existing case using the selected text.
** *Copy to clipboard* (image:images/icon-copy.png[Copy to clipboard icon,17,18]): Copy the text to clipboard to paste elsewhere. This is also helpful for resubmitting a previous prompt.
** *Add to timeline* (image:images/icon-add-to-timeline.png[Copy to clipboard icon,17,18]): Add a filter or query to Timeline using the text. This button appears for certain queries in the Security Assistant's responses.
** *Add note to timeline* (image:images/icon-add-note.png[Add note icon,16,16]): Add the selected text to your currently active Timeline as a note.
** *Add to existing case* (image:images/icon-add-to-case.png[Add to case icon,19,16]): Add a comment to an existing case using the selected text. A popup menu appears where you can select a case.
benironside marked this conversation as resolved.
Show resolved Hide resolved
** *Copy to clipboard* (image:images/icon-copy.png[Copy to clipboard icon,17,18]): Copy the text to clipboard to paste elsewhere. Also helpful for resubmitting a previous prompt.
** *Add to timeline* (image:images/icon-add-to-timeline.png[Copy to clipboard icon,17,18]): Add a filter or query to Timeline using the text. This button appears for particular queries in the AI Assistant's responses.
+
TIP: Be sure to specify which language you'd like the Security Assistant to use for queries. For example: "Can you generate an Event Query Language query to find four failed logins followed by a successful login?"
TIP: Be sure to specify which language you'd like the AI Assistant to use when writing a query. For example: "Can you generate an Event Query Language query to find four failed logins followed by a successful login?" (the intended language, "Event Query Language", is specified).
benironside marked this conversation as resolved.
Show resolved Hide resolved
** *Clear chat* (image:images/icon-clear-red.png[Red X icon,16,16]): Delete the conversation history and start a new chat.
** *Conversation settings* (image:images/icon-settings.png[Settings icon,17,17]): Choose the Generative AI connector that the Security Assistant uses, or create a new connector.

[discrete]
[[configure-ai-assistant]]
== Configure the AI Assistant
benironside marked this conversation as resolved.
Show resolved Hide resolved
The *AI Assistant settings* menu (image:images/icon-settings.png[Settings icon,17,17]) allows you to configure default conversations, quick prompts, system prompts, and data anonymization.
benironside marked this conversation as resolved.
Show resolved Hide resolved

[role="screenshot"]
image::images/assistant-settings-menu.png[The AI Assistant's settings menu, open to the Conversations tab]

The settings menu has four tabs:
joepeeples marked this conversation as resolved.
Show resolved Hide resolved

* **Conversations:** When you open the AI Assistant from certain pages, such as Timeline or Alerts, it defaults to the relevant conversation type. Choose the system prompt which appears by default for each conversation type, as well as the connector and model (if applicable).
joepeeples marked this conversation as resolved.
Show resolved Hide resolved
* **Quick Prompts:** Modify existing quick prompts or create new ones. To create a new quick prompt, type a new name in the *Name* field, then press enter. Under *Prompt*, enter or update the quick prompt's text. Under *Contexts*, select where the quick prompt should appear.
joepeeples marked this conversation as resolved.
Show resolved Hide resolved
* **System Prompts:** Edit existing system prompts or create new ones. To create a new system prompt, type a new name in the *Name* field, then press enter. Under *Prompt*, enter or update the system prompt's text. Under *Contexts*, select where the system prompt should appear.
joepeeples marked this conversation as resolved.
Show resolved Hide resolved
+
NOTE: To delete a custom prompt, open the *Name* dropdown menu, hover over the prompt you wish to delete, and click the *X* that appears to its right. You cannot delete the default prompts.
joepeeples marked this conversation as resolved.
Show resolved Hide resolved

* **Anonymization:** When you provide an event (such as an alert) to the AI Assistant as context, you can select which of its fields to include as plaintext, which to obfuscate, and which to skip altogether. The anonymization settings tab allows you to define default data anonymization behavior — you can update these settings for individual events when you include them.
benironside marked this conversation as resolved.
Show resolved Hide resolved
+
[role="screenshot"]
image::images/assistant-anonymization-menu.png[The AI Assistant's settings menu, open to the Anonymization tab]
+
The fields on this list are among those most likely to provide relevant context to the AI Assistant. Fields with *Allowed* on are included. *Allowed* fields with *Anonymized* on are included, but with their values obfuscated. When you include a particular event as context, you can use a similar interface to adjust anonymization behavior.
+
[role="screenshot"]
image::images/add-alert-context.gif[A video that shows an alert being added as context to the ]
benironside marked this conversation as resolved.
Show resolved Hide resolved
+
Be sure the anonymization behavior meets your specifications before sending a message with the event attached.

NOTE: The *Show anonymized* toggle controls whether you see the obfuscated or plaintext versions of anonymized field values in the chat interface. It does not affect what gets sent to the model provider.
benironside marked this conversation as resolved.
Show resolved Hide resolved