Skip to content

Commit

Permalink
[8.15] Updates ESS LLM recommended model matrix (backport #5622) (#5629)
Browse files Browse the repository at this point in the history
* Updates ESS LLM recommended model matrix (#5622)

* updates ESS LLM matrix

* Update docs/AI-for-security/llm-performance-matrix.asciidoc

* Update docs/AI-for-security/llm-performance-matrix.asciidoc

(cherry picked from commit 87713a8)

# Conflicts:
#	docs/AI-for-security/llm-performance-matrix.asciidoc

* fixes table

* fixes title

---------

Co-authored-by: Benjamin Ironside Goldstein <[email protected]>
Co-authored-by: Benjamin Ironside Goldstein <[email protected]>
  • Loading branch information
3 people authored Jul 30, 2024
1 parent f8d8080 commit 34c5abd
Showing 1 changed file with 8 additions and 8 deletions.
16 changes: 8 additions & 8 deletions docs/AI-for-security/llm-performance-matrix.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@

This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <<attack-discovery, Attack discovery>> or <<security-assistant, AI Assistant>>.

[cols="1,1,1,1,1,1,1", options="header"]
[cols="1,1,1,1,1,1", options="header"]
|===
| *Feature* | *Model* | | | | |
| | *Claude 3: Opus* | *Claude 3: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo*| *GPT-4 32K*

| *Assistant - General* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
| *Assistant - {esql} Generation*| Great | Great | Poor | Excellent | Poor | Excellent
| *Assistant - Alert Questions* | Excellent | Excellent | Excellent | Excellent | Poor | Good (limited context)
| *Attack discovery* | Excellent | Great | Poor | Poor | Good | Good (limited context)
| *Feature* | *Model* | | | |
| | *Claude 3: Opus* | *Claude 3.5: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo*
| *Assistant - General* | Excellent | Excellent | Excellent | Excellent | Excellent
| *Assistant - {esql} generation*| Great | Great | Poor | Excellent | Poor
| *Assistant - Alert questions* | Excellent | Excellent | Excellent | Excellent | Poor
| *Attack discovery* | Excellent | Excellent | Poor | Poor | Good
|===

0 comments on commit 34c5abd

Please sign in to comment.