feat: hogql cohort queries #27377
Open
feat: hogql cohort queries #27377
+30,854
−46,315
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… aspicer/test_change
…o aspicer/cohort-hogql
…o aspicer/cohort-hogql
…o aspicer/cohort-hogql
Co-authored-by: Michael Matloka <[email protected]>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben White <[email protected]>
Co-authored-by: Michael Matloka <[email protected]>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Paul D'Ambra <[email protected]>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
📸 UI snapshots have been updated2 snapshot changes in total. 0 added, 2 modified, 0 deleted:
Triggered by this commit. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Cohorts are still calculated using old style queries.
Changes
This code doesn't change how anything is calculated.
It adds code that runs cohort queries with new hogql queries. The new hogql queries use our existing insights as a base and compute the cohorts off of clickhouse set calculations based on those.
When we recompute a cohort, we now increment the version by 2. We run the old style cohort and insert it at x+2. We then run the new hogql style cohort and insert it at version x+1, with sign set to -1 for any old queries that still rely on sign (our deprecated way of doing versioning before argmaxing).
It runs the new code after the old code so it shouldn't interfere with cohort recalculation, and logs how long they take to run, as well as any errors that crop up.
After deploy, after a day, we will be able to find all cohorts that are failing or returning different results, and then work to debug those until the new hogql cohort calculations are good enough to become primary.
Does this work well for both Cloud and self-hosted?
Yes
How did you test this code?
Wrote a test harness and updated existing cohort tests to also run the new code and make sure that the results are the same.