chore: Track live batch exports

[tomasfarias]

Problem

We have a big advantage when it comes to auto-scaling: We can know how many batch exports will trigger at every hour, day, etc... This way, we can scale up capacity before we hit the next interval mark (i.e. the beginning of the next hour, day, etc...) as we see this number cross certain thresholds.

However, we are currently not tracking this metric.

Changes

• Add a prom gauge that:
  • Is incremented when a batch export is created or unpaused.
  • Is decremented when a batch export is deleted or paused.

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

How did you test this code?

[xvello]

I feel like this gauge will be non-trivial to use, as this timeseries will be tagged by pod, and values will fall out of scope as the web pods churn.

As we don't really need to react at the second precision, I recommend counting the relevant rows in PG and exporting that gauge from a Celery job, through the pushgateway. This way, you get a single timeseries per region, that won't suffer from aggregation artifacts.

You can clone the pg_row_count task and go from there.

[posthog-bot]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

[posthog-bot]

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.