-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Task start time conditions #105
Comments
@aphyr: what format would you suggest specifying time in and what call should we use to get the current time? |
Uh, this opens up a whole nest of issues, including that people will think the option actually means something. I'd avoid implementing this if at all possible. |
I've ran into all sorts of reasons why I've wanted to put off running a task for some period of time. What about something that says, "This can't be claimed until X seconds has passed"? It would allow for doing backoff, but wouldn't be tied to a client clock. |
Skuld's design assumptions allow for clock skew on the order of minutes to hours. Second-scale timeouts are almost definitely not feasible. |
The problem I have that I was hoping to solve with this is cases where I look at a task and see that some other resource that this task depends on isn't available right now, so I don't need to try to run this task again for another X minutes. As it is with a simple message queue (like what I use today), my only option is to throw it back into the end of the queue, which means that it will continue to see the same tasks over and over again while they're requeued until such a time that the other resource is ready again. |
You can use linear-time as a realllly loose proxy for "global time", but it can and will drift by huge volumes, and has no relation to anything outside Skuld. Heck, it may not even be close to other nodes. |
The correct solution is task dependencies. There's a writeup of this stuff in the Factual wiki; not sure if I ported it to the github repo. |
Oh, or, if your consumer is the one that discovers the dependent service is down, just have the consumer stop processing records for a bit. |
I see #15 that mentions dependencies, but I was assuming that just meant that one task has to complete before another one does. |
Tasks should be able to have a specified time that they are not started until. Tasks will not be eligible to claim until the time has passed.
The text was updated successfully, but these errors were encountered: