Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: trends query runner tests #18479

Merged
merged 15 commits into from
Nov 24, 2023
Merged

chore: trends query runner tests #18479

merged 15 commits into from
Nov 24, 2023

Conversation

Gilbert09
Copy link
Member

@Gilbert09 Gilbert09 commented Nov 8, 2023

Problem

  • We've rebuilt insights trends in HogQL, but are unsure on whether we get the same outputs from it as the old trends query

Changes

  • Copy the old trends unit tests and modified them to work on the new trends query runner
  • Ensured they all pass
    • Have added a few skips, some because we don't support certain features in HogQL land yet

How did you test this code?

  • Literal unit tests

@posthog-bot
Copy link
Contributor

Hey @Gilbert09! 👋
This pull request seems to contain no description. Please add useful context, rationale, and/or any other information that will help make sense of this change now and in the distant Mars-based future.

@Gilbert09 Gilbert09 marked this pull request as draft November 8, 2023 10:02
Base automatically changed from tom/trends-tests to master November 10, 2023 10:32
@Gilbert09 Gilbert09 force-pushed the tom/trends-old-tests branch from 76986a9 to cafd665 Compare November 10, 2023 10:37
@posthog-bot
Copy link
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

@posthog-bot posthog-bot removed the stale label Nov 23, 2023
@Gilbert09 Gilbert09 marked this pull request as ready for review November 23, 2023 13:52
@posthog-bot
Copy link
Contributor

Hey @Gilbert09! 👋
This pull request seems to contain no description. Please add useful context, rationale, and/or any other information that will help make sense of this change now and in the distant Mars-based future.

Copy link
Collaborator

@mariusandra mariusandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thoughts inline, nothing explicitly blocking. Overall great and massive work! 👍 💪

Comment on lines +135 to +136
if ContainsLazyJoinType(expression).contains_lazy_join:
return False
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copout 🤣

"""
)

compare_operators = self._compare_operators(query, "e")

assert len(compare_operators) == 1
assert compare_operators[0] == ast.CompareOperation(
left=ast.Field(chain=["event"]),
left=ast.Field(chain=["team_id"]),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come you're comparing the team_id here? This field is something that gets handled outside of HogQL... so these queries would have it double. It's in many ways reserved in HogQL (e.g. you can't make an alias with this name), so seeing it in tests makes me automatically wonder if something is wrong and if we're leaking data... 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we not intentionally exclude event from the where clause, so just wanted to pick any other field for the tests instead. But I get it being possibly weird seeing this in a test

Comment on lines +303 to +316
parse_expr(
"timestamp >= {date_from} - {inclusive_lookback}",
placeholders={
**self.query_date_range.to_placeholders(),
**self._interval_placeholders(),
},
),
parse_expr(
"timestamp <= {date_to}",
placeholders={
**self.query_date_range.to_placeholders(),
**self._interval_placeholders(),
},
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both ends of the timestamp are >= and <=. Usually I'd expect to see on bounded, and one unbounded. Bug or on purpose?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question, lemme investigate

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked, it's intentional

Comment on lines +328 to +337
filters.extend(
[
parse_expr(
"timestamp >= {date_from_with_adjusted_start_of_interval}",
placeholders=self.query_date_range.to_placeholders(),
),
parse_expr(
"timestamp <= {date_to}",
placeholders=self.query_date_range.to_placeholders(),
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question with the dateranges here

Comment on lines +179 to +182
# dateRange=DateRange(
# date_from=filter.date_from.isoformat() if filter.date_from is not None else "all",
# date_to=filter.date_to.isoformat() if filter.date_to is not None else None,
# ),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ded cod?

self.assertDictContainsSubset({"breakdown_value": "person1", "aggregated_value": 1}, event_response[0])
self.assertDictContainsSubset({"breakdown_value": "person2", "aggregated_value": 1}, event_response[1])

# TODO: test_account_filters conversion
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's to convert with them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Old comment. Lemme uncomment and figure that out. Will merge this now and get these fixes out next

@Gilbert09 Gilbert09 merged commit edd19ce into master Nov 24, 2023
71 checks passed
@Gilbert09 Gilbert09 deleted the tom/trends-old-tests branch November 24, 2023 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants