Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(scheduled flag changes): implement scheduling of changes #19395

Merged
merged 78 commits into from
Dec 21, 2023

Conversation

jurajmajerik
Copy link
Contributor

@jurajmajerik jurajmajerik commented Dec 18, 2023

Changes

Add a Celery task that populates scheduled changes on relevant models every 2 minutes. The current implementation works with Feature Flags; more models might be added in the future.

Notes on the implementation:

  • Since we have multiple concurrent Celery workers, we want to enforce concurrency=1 to prevent competing updates. To ensure this, we lock all the relevant rows with SELECT FOR UPDATE NOWAIT.
  • We process the rows in chunks to limit the memory usage. For now setting the chunk size somewhat arbitrarily to 10,000. If my back-of-the-envelope is correct, this should be on the order of 10s of megabytes of memory at most.
  • The changes specified in the payload of the scheduled_change record is populated on the model instance. If a particular update fails, log the reason for the failure in the failure_reason field of the scheduled_change record. This ensures the change is not re-processed.

How did you test this code?

Integration tests, 👀

jurajmajerik and others added 30 commits December 5, 2023 08:58
Copy link
Collaborator

@neilkakkar neilkakkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking much better :)

posthog/models/scheduled_change.py Outdated Show resolved Hide resolved
posthog/models/feature_flag/feature_flag.py Outdated Show resolved Hide resolved
posthog/models/feature_flag/feature_flag.py Show resolved Hide resolved
@jurajmajerik jurajmajerik enabled auto-merge (squash) December 21, 2023 13:01
@jurajmajerik jurajmajerik merged commit 1547f1c into master Dec 21, 2023
72 checks passed
@jurajmajerik jurajmajerik deleted the feat/schedule-feature-flag branch December 21, 2023 16:34
fuziontech pushed a commit that referenced this pull request Jan 4, 2024
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Xavier Vello <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants