Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: celery 5.x #17480

Merged
merged 5 commits into from
Sep 28, 2023
Merged

chore: celery 5.x #17480

merged 5 commits into from
Sep 28, 2023

Conversation

pauldambra
Copy link
Member

@pauldambra pauldambra commented Sep 16, 2023 •
edited
Loading

Celery needs to be updated before we can update to Python 3.11

We tried this before and some tasks appeared to stop running

Cohort calculation and clickhouse lag metrics. Running locally these tasks all trigger running this combination of versions.

Before merging

We should make sure we have a trusted metric in Prom for "tasks started" and "tasks succeeded"

Needs #17481 first to build up metrics for checking success

upgrading kombu

Current version of kombu doesn’t officially support py 3.10
and it doesn’t offiially support the version of redis that we have installed 🤷
5.3.0 is earliest we should have installed based on its changelog

### previous attempt

@pauldambra pauldambra marked this pull request as ready for review September 16, 2023 11:41
@posthog-bot
Copy link
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

@posthog-bot posthog-bot removed the stale label Sep 27, 2023
@pauldambra
Copy link
Member Author

@benjackwhite do you have any spare low context reviews?

@pauldambra
Copy link
Member Author

4.4.7 and 5.3.4 startup list are the same

[4.4.7 tasks]
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . ee.session_recordings.persistence_tasks.persist_finished_recordings
  . ee.session_recordings.persistence_tasks.persist_single_recording
  . ee.tasks.subscriptions.deliver_subscription_report
  . ee.tasks.subscriptions.handle_subscription_value_change
  . ee.tasks.subscriptions.schedule_all_subscriptions
  . posthog.api.prompt.trigger_prompt_for_user
  . posthog.celery.calculate_cohort
  . posthog.celery.calculate_decide_usage
  . posthog.celery.check_async_migration_health
  . posthog.celery.check_flags_to_rollback
  . posthog.celery.clean_stale_partials
  . posthog.celery.clear_clickhouse_deleted_person
  . posthog.celery.clickhouse_clear_removed_data
  . posthog.celery.clickhouse_errors_count
  . posthog.celery.clickhouse_lag
  . posthog.celery.clickhouse_mark_all_materialized
  . posthog.celery.clickhouse_materialize_columns
  . posthog.celery.clickhouse_mutation_count
  . posthog.celery.clickhouse_part_count
  . posthog.celery.clickhouse_row_count
  . posthog.celery.clickhouse_send_license_usage
  . posthog.celery.count_tiles_with_no_hash
  . posthog.celery.debug_task
  . posthog.celery.delete_expired_exported_assets
  . posthog.celery.demo_reset_master_team
  . posthog.celery.ee_persist_finished_recordings
  . posthog.celery.ee_persist_single_recording
  . posthog.celery.enqueue_clickhouse_execute_with_progress
  . posthog.celery.find_flags_with_enriched_analytics
  . posthog.celery.graphile_worker_queue_size
  . posthog.celery.ingestion_lag
  . posthog.celery.monitoring_check_clickhouse_schema_drift
  . posthog.celery.pg_plugin_server_query_timing
  . posthog.celery.pg_row_count
  . posthog.celery.pg_table_cache_hit_rate
  . posthog.celery.redis_celery_queue_depth
  . posthog.celery.redis_heartbeat
  . posthog.celery.schedule_all_subscriptions
  . posthog.celery.schedule_cache_updates_task
  . posthog.celery.send_org_usage_reports
  . posthog.celery.sync_all_organization_available_features
  . posthog.celery.sync_insight_cache_states_task
  . posthog.celery.sync_insight_caching_state
  . posthog.celery.update_cache_task
  . posthog.celery.update_event_partitions
  . posthog.celery.update_quota_limiting
  . posthog.celery.verify_persons_data_in_sync
  . posthog.email._send_email
  . posthog.tasks.async_migrations.run_async_migration
  . posthog.tasks.calculate_cohort.calculate_cohort_ch
  . posthog.tasks.calculate_cohort.calculate_cohort_from_list
  . posthog.tasks.calculate_cohort.clear_stale_cohort
  . posthog.tasks.calculate_cohort.insert_cohort_from_insight_filter
  . posthog.tasks.demo_create_data.create_data_for_demo_team
  . posthog.tasks.email.send_async_migration_complete_email
  . posthog.tasks.email.send_async_migration_errored_email
  . posthog.tasks.email.send_canary_email
  . posthog.tasks.email.send_email_change_emails
  . posthog.tasks.email.send_email_verification
  . posthog.tasks.email.send_fatal_plugin_error
  . posthog.tasks.email.send_invite
  . posthog.tasks.email.send_member_join
  . posthog.tasks.email.send_password_reset
  . posthog.tasks.exporter.export_asset
  . posthog.tasks.split_person.split_person
  . posthog.tasks.usage_report.capture_report
  . posthog.tasks.usage_report.send_all_org_usage_reports
  . posthog.tasks.usage_report.send_report_to_billing_service
  . posthog.tasks.user_identify.identify_task

[5.3.4tasks]
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . ee.session_recordings.persistence_tasks.persist_finished_recordings
  . ee.session_recordings.persistence_tasks.persist_single_recording
  . ee.tasks.subscriptions.deliver_subscription_report
  . ee.tasks.subscriptions.handle_subscription_value_change
  . ee.tasks.subscriptions.schedule_all_subscriptions
  . posthog.api.prompt.trigger_prompt_for_user
  . posthog.celery.calculate_cohort
  . posthog.celery.calculate_decide_usage
  . posthog.celery.check_async_migration_health
  . posthog.celery.check_flags_to_rollback
  . posthog.celery.clean_stale_partials
  . posthog.celery.clear_clickhouse_deleted_person
  . posthog.celery.clickhouse_clear_removed_data
  . posthog.celery.clickhouse_errors_count
  . posthog.celery.clickhouse_lag
  . posthog.celery.clickhouse_mark_all_materialized
  . posthog.celery.clickhouse_materialize_columns
  . posthog.celery.clickhouse_mutation_count
  . posthog.celery.clickhouse_part_count
  . posthog.celery.clickhouse_row_count
  . posthog.celery.clickhouse_send_license_usage
  . posthog.celery.count_tiles_with_no_hash
  . posthog.celery.debug_task
  . posthog.celery.delete_expired_exported_assets
  . posthog.celery.demo_reset_master_team
  . posthog.celery.ee_persist_finished_recordings
  . posthog.celery.ee_persist_single_recording
  . posthog.celery.enqueue_clickhouse_execute_with_progress
  . posthog.celery.find_flags_with_enriched_analytics
  . posthog.celery.graphile_worker_queue_size
  . posthog.celery.ingestion_lag
  . posthog.celery.monitoring_check_clickhouse_schema_drift
  . posthog.celery.pg_plugin_server_query_timing
  . posthog.celery.pg_row_count
  . posthog.celery.pg_table_cache_hit_rate
  . posthog.celery.redis_celery_queue_depth
  . posthog.celery.redis_heartbeat
  . posthog.celery.schedule_all_subscriptions
  . posthog.celery.schedule_cache_updates_task
  . posthog.celery.send_org_usage_reports
  . posthog.celery.sync_all_organization_available_features
  . posthog.celery.sync_insight_cache_states_task
  . posthog.celery.sync_insight_caching_state
  . posthog.celery.update_cache_task
  . posthog.celery.update_event_partitions
  . posthog.celery.update_quota_limiting
  . posthog.celery.verify_persons_data_in_sync
  . posthog.email._send_email
  . posthog.tasks.async_migrations.run_async_migration
  . posthog.tasks.calculate_cohort.calculate_cohort_ch
  . posthog.tasks.calculate_cohort.calculate_cohort_from_list
  . posthog.tasks.calculate_cohort.clear_stale_cohort
  . posthog.tasks.calculate_cohort.insert_cohort_from_insight_filter
  . posthog.tasks.demo_create_data.create_data_for_demo_team
  . posthog.tasks.email.send_async_migration_complete_email
  . posthog.tasks.email.send_async_migration_errored_email
  . posthog.tasks.email.send_canary_email
  . posthog.tasks.email.send_email_change_emails
  . posthog.tasks.email.send_email_verification
  . posthog.tasks.email.send_fatal_plugin_error
  . posthog.tasks.email.send_invite
  . posthog.tasks.email.send_member_join
  . posthog.tasks.email.send_password_reset
  . posthog.tasks.exporter.export_asset
  . posthog.tasks.split_person.split_person
  . posthog.tasks.usage_report.capture_report
  . posthog.tasks.usage_report.send_all_org_usage_reports
  . posthog.tasks.usage_report.send_report_to_billing_service
  . posthog.tasks.user_identify.identify_task

benjackwhite
benjackwhite approved these changes Sep 28, 2023
View reviewed changes
Copy link
Contributor

@benjackwhite benjackwhite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go assuming you're ready to keep an eye on it

@pauldambra pauldambra merged commit 2db0459 into master Sep 28, 2023
@pauldambra pauldambra deleted the chore/celery-5.0.0 branch September 28, 2023 13:01
@pauldambra pauldambra mentioned this pull request Oct 10, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benjackwhite benjackwhite benjackwhite approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pauldambra @posthog-bot @benjackwhite