[8.14] Adds LLM performance matrix doc to ESS & serverless (backport #…

…5286) (#5295) * Adds LLM performance matrix doc to ESS & serverless (#5286) * Adds LLM performance matrix doc to ESS & serverless * experimenting with matrix formatting * uses alternative matrix formatting * updates matrix format, adds alternative version * uses updated table format * updates ESS version of table * fixes table * removes outdated table formatting * Update docs/assistant/llm-performance-matrix.asciidoc Co-authored-by: Joe Peeples <[email protected]> * Update docs/serverless/assistant/llm-performance-matrix.mdx Co-authored-by: Joe Peeples <[email protected]> * Update docs/serverless/assistant/llm-performance-matrix.mdx Co-authored-by: Joe Peeples <[email protected]> * Update docs/assistant/llm-performance-matrix.asciidoc Co-authored-by: Joe Peeples <[email protected]> * Update docs/assistant/llm-performance-matrix.asciidoc Co-authored-by: natasha-moore-elastic <[email protected]> * Update docs/serverless/assistant/llm-performance-matrix.mdx Co-authored-by: natasha-moore-elastic <[email protected]> * Update docs/assistant/llm-performance-matrix.asciidoc --------- Co-authored-by: Joe Peeples <[email protected]> Co-authored-by: natasha-moore-elastic <[email protected]> (cherry picked from commit adb07fa) # Conflicts: # docs/serverless/serverless-security.docnav.json * remove serverless pages from backport branch * fixes build --------- Co-authored-by: Benjamin Ironside Goldstein <[email protected]> Co-authored-by: Benjamin Ironside Goldstein <[email protected]>
elastic · Jun 3, 2024 · 54b3b77 · 54b3b77
1 parent 2c31523
commit 54b3b77
Show file tree

Hide file tree

Showing 2 changed files with 16 additions and 0 deletions.
diff --git a/docs/assistant/llm-performance-matrix.asciidoc b/docs/assistant/llm-performance-matrix.asciidoc
@@ -0,0 +1,15 @@
+[[llm-performance-matrix]]
+= Large language model performance matrix
+
+This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <<attack-discovery, Attack discovery>> or <<security-assistant, AI Assistant>>.
+
+[cols="1,1,1,1,1,1,1", options="header"]
+|===
+| *Feature*                     | *Model*               |                    |                   |         |              |             
+|                               | *Claude 3: Opus*      | *Claude 3: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo*| *GPT-4 32K* 
+
+| *Assistant - General*         | Excellent             | Excellent          | Excellent         | Excellent | Excellent     | Excellent
+| *Assistant - {esql} Generation*| Great                 | Great              | Poor              | Excellent | Poor          | Excellent
+| *Assistant - Alert Questions* | Excellent             | Excellent          | Excellent         | Excellent | Poor          | Good (limited context)
+| *Attack discovery*            | Excellent             | Great              | Poor              | Poor      | Good          | Good (limited context)
+|===
diff --git a/docs/assistant/security-assistant.asciidoc b/docs/assistant/security-assistant.asciidoc
@@ -194,6 +194,7 @@ In addition to practical advice, AI Assistant can offer conceptual advice, tips,
 
 
 include::ai-alert-triage.asciidoc[leveloffset=+1]
+include::llm-performance-matrix.asciidoc[leveloffset=+1]
 include::azure-openai-setup.asciidoc[leveloffset=+1]
 include::connect-to-openai.asciidoc[leveloffset=+1]
 include::connect-to-bedrock.asciidoc[leveloffset=+1]