Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(max): Make summary more actionable and self-aware #27005

Merged
merged 9 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions ee/hogai/schema_generator/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,15 +106,18 @@
],
)

final_message = VisualizationMessage(
plan=generated_plan,
answer=message.query,
initiator=start_id,
id=str(uuid4()),
)
if final_message.answer.breakdownFilter:

Check failure on line 115 in ee/hogai/schema_generator/nodes.py

View workflow job for this annotation

GitHub Actions / Python code quality checks

Item "None" of "AssistantTrendsQuery | AssistantFunnelsQuery | None" has no attribute "breakdownFilter"
# Increase breakdown limit, since the "Other" series is pretty low-value for the assistant
final_message.answer.breakdownFilter.breakdown_limit = 100

Check failure on line 117 in ee/hogai/schema_generator/nodes.py

View workflow job for this annotation

GitHub Actions / Python code quality checks

Item "None" of "AssistantTrendsQuery | AssistantFunnelsQuery | None" has no attribute "breakdownFilter"

return PartialAssistantState(
messages=[
VisualizationMessage(
plan=generated_plan,
answer=message.query,
initiator=start_id,
id=str(uuid4()),
)
],
messages=[final_message],
intermediate_steps=None,
)

Expand Down
9 changes: 7 additions & 2 deletions ee/hogai/summarizer/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@
from rest_framework.exceptions import APIException
from sentry_sdk import capture_exception

from ee.hogai.summarizer.prompts import SUMMARIZER_INSTRUCTION_PROMPT, SUMMARIZER_SYSTEM_PROMPT
from ee.hogai.summarizer.prompts import (
SUMMARIZER_INSTRUCTION_PROMPT,
SUMMARIZER_SYSTEM_PROMPT,
SUMMARIZER_RESULTS_PROMPT,
)
from ee.hogai.utils.nodes import AssistantNode
from ee.hogai.utils.types import AssistantNodeName, AssistantState, PartialAssistantState
from posthog.api.services.query import process_query_dict
Expand Down Expand Up @@ -100,5 +104,6 @@ def _construct_messages(self, state: AssistantState) -> list[tuple[str, str]]:
elif isinstance(message, AssistantMessage):
conversation.append(("assistant", message.content))

conversation.append(("human", SUMMARIZER_INSTRUCTION_PROMPT))
conversation.append(("system", SUMMARIZER_RESULTS_PROMPT))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two system prompts? It's better to keep a single system prompt.

Copy link
Member Author

@Twixes Twixes Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was playing with a different approach, but actually better to consolidate. Pushed

conversation.append(("user", SUMMARIZER_INSTRUCTION_PROMPT))
return conversation
21 changes: 14 additions & 7 deletions ee/hogai/summarizer/prompts.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
SUMMARIZER_SYSTEM_PROMPT = """
Act as an expert product manager. Your task is to summarize query results in a a concise way.
Act as an expert product manager. Your task is to help the user build a successful product and business. Remember, you're a hedeghog.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a hedgehog role?😆 Do you want to force it to answer as a hedgehog?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more setting the ground for the pure conversation skill, but we definitely will want to be aware it's Max the Hedgehog rather than "expert product manager, an OpenAI large language model"

Offer actionable feedback if possible. Only provide feedback that you're absolutely certain will be useful for this team.
Assume the user is familiar with Silicon Valley terms.

The product being analyzed is described as follows:
{{product_description}}"""

SUMMARIZER_INSTRUCTION_PROMPT = """
Here are the {{query_kind}} results for this question:
SUMMARIZER_RESULTS_PROMPT = """
Here are the {{query_kind}} results for the latest question's query:
```json
{{results}}
```
```"""

SUMMARIZER_INSTRUCTION_PROMPT = """
Based on the results, answer my latest question and provide actionable feedback.
Avoid generic advice. Take into account what you know about the product.

If there are interesting trends or anomalies, succintly point them out. I can see the chart, so don't just describe all of it.
The answer needs to be high-impact, no more than a few sentences. Bullets can improve clarity of action points.
Use Silicon Valley lingo. Be informal, but without fluff. NEVER USE TITLE CASE, even in headings. Our style is sentence case always.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why didn't you include the language style instructions in the first system prompt? Or you can concatenate the second system prompt and the user prompt.


Answer my earlier question using the results above. Point out interesting trends or anomalies.
Take into account what you know about my product. If possible, offer actionable feedback, but avoid generic advice.
Limit yourself to a few sentences. The answer needs to be high-impact and relevant for me as a Silicon Valley engineer.
You MUST point out if the executed query or its results are insufficient for a full answer to my question.
"""
Loading