feat(cdp): Hog Watcher v2

[benjackwhite]

Problem

V1 was complex and hard to manage as well as having a bunch of flaws.
V2 will attempt to be slightly better

Changes

• Moved to a score based system. Means we are doing a redis request per batch which should be totally fine
• The final execution of the function is what gets observed
• Good responses score points with a gradual reduction the slower they ran. After a threshold they get a negative score
• Errors incur immediate negative points
• Once the function falls below a threshold then it moves into degraded state (overflowed) and then eventually disabled

TODO

• Fix moving to overflow
• Fix temporary disabled - likely need a second key or something (or to move this to celery or something...)

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

[posthog-bot]

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1, diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[github-actions]

Size Change: 0 B

Total Size: 1.07 MB

ℹ️ View Unchanged

Filename	Size
frontend/dist/toolbar.js	1.07 MB

_{compressed-size-action}

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 1)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

• chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.