-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(max): Make summary more actionable and self-aware #27005
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wondering if we've ever want to try something that helps us version prompts? :)
Or maybe use a Posthog tm experiment?
Size Change: 0 B Total Size: 1.11 MB ℹ️ View Unchanged
|
02a7d0c
to
6ab4d88
Compare
This reverts commit 705c1d6.
docker-compose.base.yml
Outdated
ports: | ||
- 3000:3000 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ee/hogai/summarizer/prompts.py
Outdated
@@ -1,17 +1,24 @@ | |||
SUMMARIZER_SYSTEM_PROMPT = """ | |||
Act as an expert product manager. Your task is to summarize query results in a a concise way. | |||
Act as an expert product manager. Your task is to help the user build a successful product and business. Remember, you're a hedeghog. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why a hedgehog role?😆 Do you want to force it to answer as a hedgehog?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's more setting the ground for the pure conversation skill, but we definitely will want to be aware it's Max the Hedgehog rather than "expert product manager, an OpenAI large language model"
ee/hogai/summarizer/nodes.py
Outdated
@@ -100,5 +104,6 @@ def _construct_messages(self, state: AssistantState) -> list[tuple[str, str]]: | |||
elif isinstance(message, AssistantMessage): | |||
conversation.append(("assistant", message.content)) | |||
|
|||
conversation.append(("human", SUMMARIZER_INSTRUCTION_PROMPT)) | |||
conversation.append(("system", SUMMARIZER_RESULTS_PROMPT)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two system prompts? It's better to keep a single system prompt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was playing with a different approach, but actually better to consolidate. Pushed
ee/hogai/summarizer/prompts.py
Outdated
|
||
If there are interesting trends or anomalies, succintly point them out. I can see the chart, so don't just describe all of it. | ||
The answer needs to be high-impact, no more than a few sentences. Bullets can improve clarity of action points. | ||
Use Silicon Valley lingo. Be informal, but without fluff. NEVER USE TITLE CASE, even in headings. Our style is sentence case always. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why didn't you include the language style instructions in the first system prompt? Or you can concatenate the second system prompt and the user prompt.
Makes us hit context limits!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Problem
Agent doesn't always recognize it hasn't really answered the question.
Changes
This tweaks the summarizer to be more self-aware and self-critical.
How did you test this code?
TODO!