diff --git a/docs/assistant/llm-performance-matrix.asciidoc b/docs/assistant/llm-performance-matrix.asciidoc new file mode 100644 index 0000000000..e36b7bf01d --- /dev/null +++ b/docs/assistant/llm-performance-matrix.asciidoc @@ -0,0 +1,15 @@ +[[llm-performance-matrix]] += Large language model performance matrix + +This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <> or <>. + +[cols="1,1,1,1,1,1,1", options="header"] +|=== +| *Feature* | *Model* | | | | | +| | *Claude 3: Opus* | *Claude 3: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo*| *GPT-4 32K* + +| *Assistant - General* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent +| *Assistant - {esql} Generation*| Great | Great | Poor | Excellent | Poor | Excellent +| *Assistant - Alert Questions* | Excellent | Excellent | Excellent | Excellent | Poor | Good (limited context) +| *Attack discovery* | Excellent | Great | Poor | Poor | Good | Good (limited context) +|=== diff --git a/docs/assistant/security-assistant.asciidoc b/docs/assistant/security-assistant.asciidoc index 9f0066833c..7c94ff23db 100644 --- a/docs/assistant/security-assistant.asciidoc +++ b/docs/assistant/security-assistant.asciidoc @@ -194,6 +194,7 @@ In addition to practical advice, AI Assistant can offer conceptual advice, tips, include::ai-alert-triage.asciidoc[leveloffset=+1] +include::llm-performance-matrix.asciidoc[leveloffset=+1] include::azure-openai-setup.asciidoc[leveloffset=+1] include::connect-to-openai.asciidoc[leveloffset=+1] include::connect-to-bedrock.asciidoc[leveloffset=+1] diff --git a/docs/serverless/assistant/llm-performance-matrix.mdx b/docs/serverless/assistant/llm-performance-matrix.mdx new file mode 100644 index 0000000000..bec3ea79a5 --- /dev/null +++ b/docs/serverless/assistant/llm-performance-matrix.mdx @@ -0,0 +1,19 @@ +--- +id: llm-performance-matrix +slug: /serverless/security/llm-performance-matrix +title: Large language model performance matrix +description: Learn how different models perform on different tasks in ((elastic-sec)). +tags: ["security", "overview", "get-started"] +status: in review +--- + +This table describes the performance of various large language models (LLMs) for different use cases in ((elastic-sec)), based on our internal testing. To learn more about these use cases, refer to or . + +| **Feature:** | **Model** | | | | | | +|-------------------------------|-----------------------|--------------------|--------------------|------------|-----------------|----------------| +| | **Claude 3: Opus** | **Claude 3: Sonnet** | **Claude 3: Haiku** | **GPT-4o** | **GPT-4 Turbo** | **GPT-4 32K** | +| **Assistant: general** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent | +| **Assistant: ((esql)) generation** | Great | Great | Poor | Excellent | Poor | Excellent | +| **Assistant: alert questions** | Excellent | Excellent | Excellent | Excellent | Poor | Good (limited context) | +| **Attack discovery** | Excellent | Great | Poor | Poor | Good | Good (limited context) | + diff --git a/docs/serverless/serverless-security.docnav.json b/docs/serverless/serverless-security.docnav.json index e2b0da2e04..fbc65af420 100644 --- a/docs/serverless/serverless-security.docnav.json +++ b/docs/serverless/serverless-security.docnav.json @@ -30,6 +30,9 @@ { "id":"serverlessSecurityAssistantAlertTriage" }, + { + "id":"llm-performance-matrix" + }, { "id": "serverlessSecurityConnectBedrock" },