-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Batch Support #142
Draft
jpcamara
wants to merge
15
commits into
rails:main
Choose a base branch
from
jpcamara:batch-poc
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Batch Support #142
Commits on Sep 26, 2024
-
* Introduces a "batch" concept, similar to batches present in Sidekiq Pro and GoodJob * Batches monitor a set of jobs, and when those jobs are completed can fire off a final job * This introduces a SolidQueue::JobBatch model, as well as the ability to enqueue jobs and associate them with the batch * There are still more ideas to figure out, but this provides a basic batch scaffolding to spark discussion
Configuration menu - View commit details
-
Copy full SHA for db7d651 - Browse repository at this point
Copy the full SHA db7d651View commit details -
Configuration menu - View commit details
-
Copy full SHA for 308775e - Browse repository at this point
Copy the full SHA 308775eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 52086ac - Browse repository at this point
Copy the full SHA 52086acView commit details -
Allow batch jobs to be instances
* This means we can store the arguments and settings by letting the user do `BatchJob.new(arguments).set(options)` * Yield the batch in `enqueue` in case someone needs info from it * When you serialize then deserialize an activejob instance, the arguments are in the serialized_arguments field and can only be transferred over by the private method `deserialize_arguments_if_needed`. This is pretty janky, so there is probably something i'm missing * `perform_all_later` let's us do a perform_later even with instance, which does not seem to be possible on the instances themselves
Configuration menu - View commit details
-
Copy full SHA for 1057152 - Browse repository at this point
Copy the full SHA 1057152View commit details -
Make sure
batch
is still first arg of the batch callback* Add spec for adding arguments and options to the batch callback
Configuration menu - View commit details
-
Copy full SHA for c8d784d - Browse repository at this point
Copy the full SHA c8d784dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e49ee43 - Browse repository at this point
Copy the full SHA e49ee43View commit details -
* Spacing, double quotes * Support Ruby < 3.2 by removing the implicit key/variable syntax
Configuration menu - View commit details
-
Copy full SHA for b8badaa - Browse repository at this point
Copy the full SHA b8badaaView commit details -
Handle on_failure and on_success
* on_failure fires the first time any of the jobs fail, even once * on_success only fires if all jobs work (after retries) * remove unused job_id
Configuration menu - View commit details
-
Copy full SHA for 0f16d1c - Browse repository at this point
Copy the full SHA 0f16d1cView commit details -
Allow enqueueing into a batch instance
* Allows enqueueing a job within a job, as part of the batch
Configuration menu - View commit details
-
Copy full SHA for 9cd7d5b - Browse repository at this point
Copy the full SHA 9cd7d5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for b494353 - Browse repository at this point
Copy the full SHA b494353View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7c1c31d - Browse repository at this point
Copy the full SHA 7c1c31dView commit details -
* Parent batches will not complete until all child batches have been completed
Configuration menu - View commit details
-
Copy full SHA for 07821d6 - Browse repository at this point
Copy the full SHA 07821d6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1a60a9e - Browse repository at this point
Copy the full SHA 1a60a9eView commit details -
Configuration menu - View commit details
-
Copy full SHA for b4d3f8d - Browse repository at this point
Copy the full SHA b4d3f8dView commit details -
* Attach success jobs to the parent batch, not to the current batch (which has already finished at this point)
Configuration menu - View commit details
-
Copy full SHA for 1a0846a - Browse repository at this point
Copy the full SHA 1a0846aView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.