Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(data-warehouse): Added the ability to use dashboard filters in BI queries #24033

Merged
merged 10 commits into from
Jul 31, 2024

Conversation

Gilbert09
Copy link
Member

@Gilbert09 Gilbert09 commented Jul 26, 2024

Problem

  • We wanna be able to dashboard filters in BI/warehouse queries

Changes

  • This only enables the usage of the date range filters right now. A more complete filtering experience is to come next week (stay tuned!)
  • Allows the usage of {filters.dateRange.from} and {filters.dateRange.to} in SQL queries when comparing with a datetime
  • If dateTo is not set, then the comparison node gets effectively removed

@Twixes @mariusandra some review of my C++ is code is very much needed please!! I've tried to join a vector<string> with a delimiter (.) - I got to this code from googling and chatgpt, but please do correct me if I've done something horrifically wrong in the C++ world 🤓

Usage:

select * from prod_stripe_customers
where created_at >= {filters.dateRange.from} and created_at < {filters.dateRange.to}

TODO:

  • Need to add tests before this gets merged - throwing this up early to get some 👀 on it

Does this work well for both Cloud and self-hosted?

Yes

How did you test this code?

Tests to be added before this merges

@Gilbert09 Gilbert09 requested review from mariusandra and a team July 26, 2024 16:53
@Gilbert09 Gilbert09 changed the title feat(data-warehouse): Added the ability to use individual date range filters in BI queries feat(data-warehouse): Added the ability to use dashboard filters in BI queries Jul 26, 2024
@Gilbert09 Gilbert09 temporarily deployed to pypi-hogql-parser July 26, 2024 17:11 — with GitHub Actions Inactive
Copy link
Member

@EDsCODE EDsCODE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python looks good

Comment on lines 79 to 89
if "filters" in placeholders_in_query or any(
placeholder.startswith("filters.") for placeholder in placeholders_in_query
):
select_query = replace_filters(select_query, filters, team)
placeholders_in_query.remove("filters")

removed_placeholders: list[str] = []
for placeholder in placeholders_in_query:
if placeholder == "filters":
removed_placeholders.append(placeholder)
elif placeholder.startswith("filters."):
removed_placeholders.append(placeholder)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We anyway iterate over placeholders_in_query at least once in this block, so why not rewrite to only do it once, skipping over the irrelevant placeholders?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could still rewrite this block only have for placeholder in placeholders_in_query once, but that's minor

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, yep, missed this!

posthog/hogql/parser.py Outdated Show resolved Hide resolved
posthog/hogql/filters.py Outdated Show resolved Hide resolved
@Gilbert09 Gilbert09 requested a review from Twixes July 31, 2024 09:58
@posthog-bot
Copy link
Contributor

It looks like the code of hogql-parser has changed since last push, but its version stayed the same at 1.0.33. 👀
Make sure to resolve this in hogql_parser/setup.py before merging!

@Gilbert09 Gilbert09 temporarily deployed to pypi-hogql-parser July 31, 2024 10:18 — with GitHub Actions Inactive
Copy link
Member

@Twixes Twixes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple bits left that would be nice to clean up, but overall :shipit:

hogql_parser/parser.cpp Outdated Show resolved Hide resolved
Comment on lines 79 to 89
if "filters" in placeholders_in_query or any(
placeholder.startswith("filters.") for placeholder in placeholders_in_query
):
select_query = replace_filters(select_query, filters, team)
placeholders_in_query.remove("filters")

removed_placeholders: list[str] = []
for placeholder in placeholders_in_query:
if placeholder == "filters":
removed_placeholders.append(placeholder)
elif placeholder.startswith("filters."):
removed_placeholders.append(placeholder)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could still rewrite this block only have for placeholder in placeholders_in_query once, but that's minor

Co-authored-by: Michael Matloka <[email protected]>
@posthog-bot
Copy link
Contributor

It looks like the code of hogql-parser has changed since last push, but its version stayed the same at 1.0.34. 👀
Make sure to resolve this in hogql_parser/setup.py before merging!

@Gilbert09 Gilbert09 temporarily deployed to pypi-hogql-parser July 31, 2024 11:44 — with GitHub Actions Inactive
@Gilbert09 Gilbert09 merged commit 59a1523 into master Jul 31, 2024
88 checks passed
@Gilbert09 Gilbert09 deleted the tom/dashboard-date-filters branch July 31, 2024 12:20
@Gilbert09 Gilbert09 mentioned this pull request Jul 31, 2024
69 tasks
fuziontech added a commit that referenced this pull request Jul 31, 2024
…into delete_test_fixes

* 'delete_test_fixes' of https://github.com/PostHog/posthog: (22 commits)
  chore(deps): Update posthog-js to 1.153.0 (#24113)
  fix: series with different current and previous breakdowns are throwing errors  (#24092)
  chore(deps): Update posthog-js to 1.152.1 (#24110)
  fix(bi): Use a better tooltip mode for graphs (#24101)
  feat(bi): Added support for decimal numbers in BI (#24103)
  fix: Site app url for early access (#24104)
  feat(cdp): Added mailjet templates (#24096)
  fix: dry run output was inaccurate (#24075)
  feat: Add batch utilization metric for blobby (#24094)
  chore(deps): Update posthog-js to 1.151.2 (#24106)
  fix: limit metadata reloading (#24099)
  chore: remove useless draft files (#24102)
  feat(data-warehouse): Added the ability to use dashboard filters in BI queries (#24033)
  chore(deps): Update posthog-js to 1.151.1 (#24098)
  fix: Pipeline activity logs (#24077)
  feat(BI): Series settings (#24082)
  feat(dev): add separate env var for disabling navigation hooks (#24097)
  fix(web-analytics): Fix empty channel types (#24087)
  fix: Make error contact point to support modal (#24048)
  feat(notebooks): add error boundary around components (#24095)
  ...
silentninja pushed a commit to silentninja/posthog that referenced this pull request Aug 8, 2024
…I queries (PostHog#24033)

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Matloka <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants