Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(hogql): Expression Fields & PoEv2 #18811

Merged
merged 48 commits into from
Nov 27, 2023
Merged

feat(hogql): Expression Fields & PoEv2 #18811

merged 48 commits into from
Nov 27, 2023

Conversation

mariusandra
Copy link
Collaborator

@mariusandra mariusandra commented Nov 22, 2023

Dependent on #18725

Problem

We need support for custom SQL expressions in HogQL tables.

  1. Accessing person_id on events with PoEv2 has to evaluate to something like override.override_person_id or events.person_id
  2. We'll want custom source and other columns for web analytics.
  3. We eventually want to offer calculated properties as a first class feature to users.

Changes

  • Adds a new database field type, "Expression Fields", which pretend to be regular database fields, but swap out their contents when resolving under ClickHouse.
  • Updates "person_id" and person.id when PoEv2 is enabled, makes it use the override vs old_id code.
  • Wiring to make this all work.

Known limitation

The current expression fields have no concept of this. With events.a = event, when running select distinct_id as event, a from events, we'd return the distinct_id twice, instead of a resolving to the event field, which was the only one in scope when the expression field was defined.

This will be fixed in a followup, but is non-blocking today. The biggest problem this can cause is a "error redefining alias" message or an otherwise erroring query, such as not knowing which person_id to choose if there are many in scope.

Effectively this means we can play around with these custom fields, but we shouldn't release a "calculated properties" feature to users today.

How did you test this code?

Wrote tests. Tested the new lifecycle query with PoEv2 enabled, everything just worked on the first go. It was magical.

@mariusandra mariusandra changed the title Column expressions feat(hogql): column expressions Nov 22, 2023
@mariusandra mariusandra changed the title feat(hogql): column expressions feat(hogql): column expressions & PoEv2 Nov 23, 2023
@mariusandra mariusandra changed the title feat(hogql): column expressions & PoEv2 feat(hogql): Expression Fields & PoEv2 Nov 23, 2023
@mariusandra mariusandra marked this pull request as ready for review November 23, 2023 15:56
Copy link
Member

@Gilbert09 Gilbert09 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Base automatically changed from hogql-hidden-aliases to master November 27, 2023 11:20
@mariusandra mariusandra merged commit 2586d27 into master Nov 27, 2023
71 checks passed
@mariusandra mariusandra deleted the column-expressions branch November 27, 2023 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants