Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Use max_bytes_before_external_group_by for funnels and path que… #23022

Merged
merged 13 commits into from
Jun 18, 2024

Conversation

timgl
Copy link
Collaborator

@timgl timgl commented Jun 17, 2024

…ries

Problem

Some queries fail with OOM errors. They don't actually have to fail, it could just swap to disk instead.

Changes

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

Copy link

sentry-io bot commented Jun 17, 2024

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: posthog/hogql_queries/insights/funnels/funnels_query_runner.py

Function Unhandled Issue
calculate ValidationError: 1 validation error for HogQLQueryResponse posthog.tasks.tasks.process_...
Event Count: 28
calculate ValidationError: ["Filter parameter funnel_to_step can only be one of 1 for time to convert!"] ...
Event Count: 17
calculate Cohort.DoesNotExist: Cohort matching query does not exist. posthog.tasks.tasks.process_...
Event Count: 4
calculate IndexError: list index out of range posthog.tasks...
Event Count: 2
calculate ValidationError: ["Filter parameter funnel_to_step can only be one of 1, 2 for time to convert!"] ...
Event Count: 1
📄 File: posthog/hogql_queries/insights/paths_query_runner.py (Click to Expand)
Function Unhandled Issue
calculate UnboundLocalError: local variable 'span' referenced before assignment posthog.tasks.tas...
Event Count: 1
---

Did you find this useful? React with a 👍 or 👎

@timgl timgl requested review from tkaemming and fuziontech June 17, 2024 15:26
Copy link
Member

@fuziontech fuziontech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checks out ✅
🚢 it

@timgl timgl enabled auto-merge (squash) June 18, 2024 09:14
@timgl timgl merged commit 6fd86f7 into master Jun 18, 2024
82 checks passed
@timgl timgl deleted the max-bytes-before-external-group-by branch June 18, 2024 12:57
timgl added a commit that referenced this pull request Jun 18, 2024
#23022)

* perf: Use max_bytes_before_external_group_by for funnels and path queries

* Update query snapshots

* Update query snapshots

* Update query snapshots

* Update query snapshots

* Update query snapshots

* Update query snapshots

* fix test

* also do retention query

* Update query snapshots

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
@thecatontheflat
Copy link

It seems like there is no way to enforce the "spill to disk" behavior anymore, because it just defaults to 0 in many cases now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants