feat(web-analytics): Add web analytics tabs #17981

robbie-c · 2023-10-14T09:55:40Z

Problem

Want to be able to breakdown by more properties

Ticks off 4 boxes here https://docs.google.com/document/d/1LsK70o5CTPKHWEeB_8txmJHmOMhNPCc5GnHhw87nLYY/edit

Changes

Add a new query type which does pageviews, uniques, and bounce rate, with a breakdown_by enum. This is the basis for most web analytics tables now.
Remove the unused top_sources and top_pages query types
Add a change to property_to_expr to support nested properties

How did you test this code?

(Behind FF) ran it manually

robbie-c · 2023-10-14T10:04:08Z

frontend/src/queries/schema.ts

@@ -582,15 +569,28 @@ export interface WebTopClicksQueryResponse extends QueryResponse {
    columns?: unknown[]
 }

-export interface WebTopPagesQuery extends WebAnalyticsQueryBase {
-    kind: NodeKind.WebTopPagesQuery
+export enum WebStatsBreakdown {


I used an enum here because some properties will take special handling on the query side, so I can't just pass a property name (well I could, but I'd have to handle it with cases in the same way)

robbie-c · 2023-10-14T10:14:59Z

posthog/hogql/property.py

@@ -129,7 +129,7 @@ def property_to_expr(
                return ast.Or(exprs=exprs)

        chain = ["person", "properties"] if property.type == "person" and scope != "person" else ["properties"]
-        field = ast.Field(chain=chain + [property.key])
+        field = ast.Field(chain=chain + property.key.split("."))


Drawing attention to this, theoretically could be breaking as it unescapes any ..

i.e. a.b was previously "properties"."a.b" and now will be "properties"."a"."b"

Hmm, sorry, got to block this, as it's a global change in the hogql folder. We have users who have actual dots in their property name strings, and this will break some filter somewhere for them. The event/feature/property filter is made to work with the current taxonomic filter, which presents just the first layer of properties and has never supported more.

You can use the hogql template helper like

hogql`SELECT properties.${hogql.identifier('odd property')} FROM events`

if you want to cleanly escape multiple properties into a hogql string.

This is fixed by creating hogql filters on the front end

mariusandra

All good for most of the changes, but the one in hogql/property.py will break the app for some existing users, so we're going to have to find another way.

mariusandra · 2023-10-16T08:21:31Z

posthog/hogql/property.py

@@ -129,7 +129,7 @@ def property_to_expr(
                return ast.Or(exprs=exprs)

        chain = ["person", "properties"] if property.type == "person" and scope != "person" else ["properties"]
-        field = ast.Field(chain=chain + [property.key])
+        field = ast.Field(chain=chain + property.key.split("."))


Hmm, sorry, got to block this, as it's a global change in the hogql folder. We have users who have actual dots in their property name strings, and this will break some filter somewhere for them. The event/feature/property filter is made to work with the current taxonomic filter, which presents just the first layer of properties and has never supported more.

You can use the hogql template helper like

hogql`SELECT properties.${hogql.identifier('odd property')} FROM events`

if you want to cleanly escape multiple properties into a hogql string.

robbie-c · 2023-10-16T12:43:01Z

I switched the parser over to CPP and moved the navbar icon to the right section

mariusandra

One thought regarding escaping strings, but LGTM.

mariusandra · 2023-10-17T13:03:19Z

frontend/src/scenes/web-analytics/webAnalyticsLogic.ts

 export const initialWebAnalyticsFilter = [] as WebAnalyticsPropertyFilters

+const setOncePropertyNames = ['$initial_pathname', '$initial_referrer', '$initial_utm_source', '$initial_utm_campaign']
+const hogqlForSetOnceProperty = (key: string, value: string): string => `properties.$set_once.${key} = '${value}'`


This will work for a limited subset of keys, but you can use hogql.identifier(key) (from import { hogql } from './utils') to actually safely escape all keys.

* Add web analytics tabs * Add stats table query runner and use it for pathname * Add stats query runner, replacing page and source * Remove classes for now-unused queries * Add remaining breakdowns * Prefer unknown to any * Remove unused ctes * Fix typing issues * Fix python typing * Switch to hogql filters for set_once properties * Use cpp parser * Move web analytics to the right navbar section * Fix typing issues

robbie-c changed the title ~~Add web analytics tabs~~ feat(web-analytics): Add web analytics tabs Oct 14, 2023

robbie-c commented Oct 14, 2023

View reviewed changes

robbie-c requested a review from mariusandra October 16, 2023 08:11

robbie-c marked this pull request as ready for review October 16, 2023 08:11

robbie-c requested a review from Gilbert09 October 16, 2023 08:12

mariusandra requested changes Oct 16, 2023

View reviewed changes

robbie-c added 11 commits October 16, 2023 12:01

Add web analytics tabs

e0bbd34

Add stats table query runner and use it for pathname

27b2af6

Add stats query runner, replacing page and source

1c272a2

Remove classes for now-unused queries

415116a

Add remaining breakdowns

1ce3437

Prefer unknown to any

5e69188

Remove unused ctes

5bb0e73

Fix typing issues

06ced1a

Fix python typing

295bdca

Switch to hogql filters for set_once properties

d1bc162

Use cpp parser

2e3aa89

robbie-c force-pushed the add-web-analytics-tabs branch from 206fb13 to 2e3aa89 Compare October 16, 2023 11:06

Move web analytics to the right navbar section

5de1214

robbie-c force-pushed the add-web-analytics-tabs branch from 2e9cac2 to 30babc3 Compare October 16, 2023 12:02

Fix typing issues

7c89fd6

robbie-c force-pushed the add-web-analytics-tabs branch from 30babc3 to 7c89fd6 Compare October 16, 2023 12:07

robbie-c requested a review from mariusandra October 16, 2023 12:42

robbie-c enabled auto-merge (squash) October 16, 2023 14:27

mariusandra approved these changes Oct 17, 2023

View reviewed changes

robbie-c merged commit 1fb0548 into master Oct 17, 2023
73 checks passed

robbie-c deleted the add-web-analytics-tabs branch October 17, 2023 13:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(web-analytics): Add web analytics tabs #17981

feat(web-analytics): Add web analytics tabs #17981

robbie-c commented Oct 14, 2023 •

edited

Loading

robbie-c Oct 14, 2023 •

edited

Loading

robbie-c Oct 14, 2023 •

edited

Loading

mariusandra Oct 16, 2023 •

edited

Loading

robbie-c Oct 16, 2023

mariusandra left a comment

mariusandra Oct 16, 2023 •

edited

Loading

robbie-c commented Oct 16, 2023

mariusandra left a comment

mariusandra Oct 17, 2023

feat(web-analytics): Add web analytics tabs #17981

feat(web-analytics): Add web analytics tabs #17981

Conversation

robbie-c commented Oct 14, 2023 • edited Loading

Problem

Changes

How did you test this code?

robbie-c Oct 14, 2023 • edited Loading

Choose a reason for hiding this comment

robbie-c Oct 14, 2023 • edited Loading

Choose a reason for hiding this comment

mariusandra Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

robbie-c Oct 16, 2023

Choose a reason for hiding this comment

mariusandra left a comment

Choose a reason for hiding this comment

mariusandra Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

robbie-c commented Oct 16, 2023

mariusandra left a comment

Choose a reason for hiding this comment

mariusandra Oct 17, 2023

Choose a reason for hiding this comment

robbie-c commented Oct 14, 2023 •

edited

Loading

robbie-c Oct 14, 2023 •

edited

Loading

robbie-c Oct 14, 2023 •

edited

Loading

mariusandra Oct 16, 2023 •

edited

Loading

mariusandra Oct 16, 2023 •

edited

Loading