Replies: 1 comment
-
Hi, |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
First mistake, first big bill ;)
I've launched a migration on production, that put +70k jobs in the queue.
AWS + Lambda is a good boy, it did started to handle the jobs : each job had to query DB, delete some records and reinsert some new records...
Lambda scaled up to 1000 concurrent jobs, and DB (planetscale hobbie plan) has been hammered with 1000 concurrent connexions. Soon, it started to reject connections, and performance got worse and worse (+100s for a job to execute, since of DB not being able to handle 1000 DELETE on the same table). This leaded to a 75$ bill, and could have been worse, because it purged the queue half way....
TLDR : Is there a (easy) way to set a limit on concurrent execution of lambdas ?
Beta Was this translation helpful? Give feedback.
All reactions