-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Remove promise manager usage from ingestion warnings & db #17650
Conversation
a5fb70e
to
c0e6164
Compare
Interesting, I hadn't run into Behaviorally the diff makes sense to me, but it does seem like we'll now block and |
yes, potentially, but our ingestion parallelism should handle this, no? Also afaik there's two options: someone somewhere waits for it or we drop it occasionally. Wouldn't be surprised if we did the latter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, let's nuke this manager 🔥
nit for possibly later
: I'm wondering whether we really need the delivery guarantee though: await
ing ensures we never lose an ingestion warning on pod crash, but do we really need this guarantee? I feel like we could get away with firing off captureIngestionWarning
and not await
ing it.
captureIngestionWarning
returns once the message is in the rdkafka producer queue, that we expect to be properly flushed on graceful shutdown (we await kafkaProducer.disconnect()
on closeHub
), so the only data loss scenario is pod crash/oom.
plugin-server/src/main/ingestion-queues/batch-processing/each-batch-ingestion-kafkajs.ts
Outdated
Show resolved
Hide resolved
plugin-server/src/main/ingestion-queues/batch-processing/each-batch-ingestion.ts
Outdated
Show resolved
Hide resolved
There was something about voided promises that was the reason, promiseManager was added in the first place iirc ... too many voided promises cause some crashes / stalling 🤷♀️ |
c252fc0
to
86ddb3a
Compare
Problem
Promise manager adds confusion to how things work and could be causing the ingestion pipeline to stall at points, lets nuke it.
Changes
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
How did you test this code?