Skip to content

Commit

Permalink
chore: add note about app_metrics aggregation (#19449)
Browse files Browse the repository at this point in the history
  • Loading branch information
bretthoerner authored Dec 20, 2023
1 parent a8fc6ec commit 0f896a9
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion posthog/models/app_metrics/sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@
error_details String CODEC(ZSTD(3))
""".strip()

# NOTE: We have producers that take advantage of the timestamp being truncated to the hour,
# i.e. they batch up metrics and send them pre-truncated. If we ever change this truncation
# we need to revisit producers (e.g. the webhook service currently known as rusty-hook or pgqueue).
APP_METRICS_TIMESTAMP_TRUNCATION = "toStartOfHour(timestamp)"

APP_METRICS_DATA_TABLE_SQL = (
lambda: f"""
CREATE TABLE IF NOT EXISTS sharded_app_metrics ON CLUSTER '{settings.CLICKHOUSE_CLUSTER}'
Expand All @@ -35,7 +40,7 @@
)
ENGINE = {SHARDED_APP_METRICS_TABLE_ENGINE()}
PARTITION BY toYYYYMM(timestamp)
ORDER BY (team_id, plugin_config_id, job_id, category, toStartOfHour(timestamp), error_type, error_uuid)
ORDER BY (team_id, plugin_config_id, job_id, category, {APP_METRICS_TIMESTAMP_TRUNCATION}, error_type, error_uuid)
"""
)

Expand Down

0 comments on commit 0f896a9

Please sign in to comment.