Hog CDP Megaissue

[benjackwhite]

Hog CDP Megaissue

• MVP feat: DeliveryHog #22706
• Hog specific TODOs: Hog/QL tasklist meta#208

Closed beta release

• Callback service for handling async function responses feat: DeliveryHog callback consumer #22884
• Proper templates - allow them to be created and saved. Potentially also move to the backend so that we can temporarily enforce that one can only use templates (until we are ready to allow full code modifications) - feat: CDP Templates #22896
• Icon generation / uploading - feat: Automatic Hog Function icons #22923
• Add and test a bunch of templates feat: More Hog templates #22957
  • Hubspot
  • Customer.io
  • Generic Webhook
• Limit to a max of 2 async functions for now feat: Refactor Hog executor #22977
• UX tidying - icons, display in list etc.
• Move bytecode refreshing to background job
• Test execution with sample event - feat: Test hog functions #23017
• Scaling work
  • Overflow topic with detection for slow functions - do it all in redis - feat(cdp): HogWatcher #23286
  • Metrics and dashboards feat(cdp): Add a bunch more metrics counting for hog functions #23281
  • Alerts for key metrics
• Add indicator and info about the new functions
  • Banner when configuring
  • "Coming soon" for app metrics
  • Filtering destinations based on type (batch exports, legacy realtime, new realtime)
• Hook up to an actual webhook service
• Load testing of Hog functions in our own account
• Deploy in the EU
• Hog parsing with c++

Open beta release (feature preview)

• Get LogEntries connected
• Templating helpers - find a relevant event and give autocomplete suggestions for both string and code inputs
• Get AppMetrics connected feat(cdp): App metrics #23893
• Native oauth integration (needed for a lot of destinations, e.g. salesforce)
• Test autocapture work (likely works but lets just triple check)
• Make Slack destination free feat: Make Slack destination free #23880

General release

• Secret management
• Fix activity log
• Redact the payload responses when testing
• Test event data from real events
• Event throughput estimation - show a rough estimate of how many matching events will trigger with warnings
• Ensure that the billing guard is in place for data pipelines addon
• Indicate when oauth integration scopes need refreshing
• Fix groups stuff as that is broken now...
• Hide plugins that we have replaced
• Ensure actions cannot have cohorts for filtering
• Setup unlayer account
• Migrate action webhooks
  • Migrate over without changing pricing
  • Make generic webhooks free?
  • Setup new usage based pricing
• Migrate zapier hooks
• 1-1 parity with existing destinations with automated migrations
• Update webhooks docs to talk about hog functions
• Update docs for Hog Functions
• Adding hog function executions to usage reports

Non-blocking ideas / work

• Show function status in the list view
• ❗ Activity log stream when function state changes
• ❗ Alert when function fails feat: Move function transition tracking into celery #25173
• Linked destinations for actions feat(cdp): Easy setup of destinations from other sources #24394
• Make "internal" hog functions that are perhaps managed separately with way more limited code options
• Fix channel selector to save the name as well if possible
• Ensure optional properties work as expected person?.properties?.email
• Expose posthog.capture method as a minimum so we can have more control over these use cases
• Remove old action matching bytecodes from postgres
• Indicate specifically if the underlying template has been updated since it was forked (allowing people to know there are "updates" available
• "Masking" concept - have a way of indicating "this function should only be run once per X"
• Setup functions so that you explicitly choose to deviate from the template rather than always deviating

Plugin parity

List of existing destinations and whether we have a corresponding Hog solution. Tracked in more detail here.

Migration plan

Action webhooks

• Once we have things stable, we can run a migration script to transform the payloads. Given that the message formatting is relatively simplistic this should be totally doable

Zapier "hooks"

• Zapier has a special endpoint for creating these "hooks". We could likely consider continuing to use the API for simplicities sake but create dedicated destinations instead (this gives other ways to customize / disable it and see delivery metrics)

Plugins

• These we need to do on a case by case basis. Some are 1to1 transferrable, others have slight changes that the user might need to be aware of (we can do this with in app messaging)