Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: cache cancellations to prevent race conditions in pod creation #230

Merged
merged 1 commit into from
Sep 20, 2023

Conversation

shreddedbacon
Copy link
Member

@shreddedbacon shreddedbacon commented Sep 19, 2023

Checklist

  • Affected Issues have been mentioned in the Closing issues section
  • Documentation has been written/updated
  • PR title is ready for changelog and subsystem label(s) applied

There exists a condition where a cancellation can be actioned before the task or build has been started, so that when the task or build are started they have no idea that it was cancelled. This is because there are two queues used, one for builds and tasks, the other for misc tasks. Cancellations enter via the misc queue, and because they are handled in their own queue, it is possible that the cancellation message could be processed before the build or task message is handled completely.

This implements an expiring cache that will cache cancellations, so that if the condition is met and the pod gets started, the pod controller will check the cache periodically and if the cancellation exists in cache will cancel the task or build in the way it would normally be cancelled.

@shreddedbacon shreddedbacon force-pushed the cancel-cache branch 6 times, most recently from 01d9496 to d028e48 Compare September 19, 2023 09:40
@shreddedbacon shreddedbacon marked this pull request as ready for review September 19, 2023 21:58
@shreddedbacon shreddedbacon merged commit 900dcba into main Sep 20, 2023
9 checks passed
@shreddedbacon shreddedbacon deleted the cancel-cache branch September 20, 2023 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant