Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wip: add support for ingesting events via pubsub #1044

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

bhearsum
Copy link
Contributor

@bhearsum bhearsum commented Jun 24, 2024

When completed, this should add support for consuming pubsub messages from GCS product delivery buckets. I suggest we continuing running both this and SQS at first, and then:

  • Switch off buildhub uploads to SQS
  • Verify that buildhub still gets all the data it needs
  • Turn off & rip out SQS support in buildhub

This actually should be quite easy - the pubsub API is pretty straightforward and compatible with the current architecture. The tricky part is that buildhub is woefully out of date, which makes adding new dependencies terribly difficult. The tests also don't seem to run without a postgres database being available, which makes the development cycle annoying and more difficult.

I suggest we try to bring buildhub up to date before tackling this any further.

@bhearsum
Copy link
Contributor Author

#1045 and #1046 for bringing the Python side of things up to speed, #1047 for test improvements.

@bhearsum
Copy link
Contributor Author

I also spun the Python 3.12 upgrade into #1048

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant