perf: Speed up filtering persons #25604
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
📸 UI snapshots have been updated2 snapshot changes in total. 0 added, 2 modified, 0 deleted:
Triggered by this commit. |
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
📸 UI snapshots have been updated2 snapshot changes in total. 0 added, 2 modified, 0 deleted:
Triggered by this commit. |
📸 UI snapshots have been updated3 snapshot changes in total. 0 added, 3 modified, 0 deleted:
Triggered by this commit. |
📸 UI snapshots have been updated3 snapshot changes in total. 0 added, 3 modified, 0 deleted:
Triggered by this commit. |
Suspect IssuesThis pull request was deployed and Sentry observed the following issues:
Did you find this useful? React with a 👍 or 👎 |
fuziontech
added a commit
that referenced
this pull request
Oct 21, 2024
* master: (100 commits) revert: perf: Speed up filtering persons (#25715) chore: count data ingested by source (#25714) fix(bi): Dont try to be too clever with variables (#25712) fix(cdp): Ask for kinesis stream name, not the ARN (#25711) fix(batch-exports): Deal with deeply nested events (#25710) fix(batch-exports): Ingest as JSON in Redshift super type (#25709) perf: Speed up filtering persons (#25604) feat(data-warehouse): Add button to cancel batch export backfill run (#25706) feat(cdp): add discord template (#25649) feat(max): Show query summary (#25696) feat(errors): Use exception list and remove stack trace raw (#25655) fix: smaller cohort resource usage for replay filters (#25699) chore: use meta columns in data visualization table (#25697) feat(cdp): add klaviyo templates (#25693) feat(max): Answer feedback buttons (#25670) feat(cdp): add google ads integration warning (#25698) feat(hog): print ast in console (#25608) feat(data-warehouse): edit SQL based import configs (#25685) fix(insights): handle negation property in cohort property filters (#25691) fix: warming and filter (#25674) ...
timgl
added a commit
that referenced
this pull request
Oct 25, 2024
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Tomás Farías Santana <[email protected]> Co-authored-by: Marius Andra <[email protected]> Co-authored-by: Georgiy Tarasov <[email protected]> Co-authored-by: David Newell <[email protected]> Co-authored-by: PostHog Bot <[email protected]> Co-authored-by: Richard Borcsik <[email protected]> Co-authored-by: Michael Matloka <[email protected]> Co-authored-by: Eric Duong <[email protected]> Co-authored-by: Paul D'Ambra <[email protected]> Co-authored-by: Juraj Majerik <[email protected]> Co-authored-by: Phani Raj <[email protected]> Co-authored-by: Tom Owers <[email protected]> Co-authored-by: Dylan Martin <[email protected]> Co-authored-by: Oliver Browne <[email protected]> Co-authored-by: Marcus Hof <[email protected]> Co-authored-by: Michael Matloka <[email protected]> Co-authored-by: Anirudh Pillai <[email protected]> Co-authored-by: Sandy Spicer <[email protected]> Co-authored-by: Neil Kakkar <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Currently, we first group/argmax every single person, then we filter that result set down. It'd be much faster to filter first, then group by.
Changes
Filter first, then group by, then filter again to make sure the results are still correct.
The resulting query is a bit ugly (basically 3 sub queries) but it takes the filtering of the persons page for researchgate down from 80-90 seconds to 2 seconds!
Also removed the personsArgMaxVersion and optimizeJoinedFilters modifiers as those were making things more complicated and weren't relevant anymore.
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Does this work well for both Cloud and self-hosted?
How did you test this code?