Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KS-87] Generic workflow engine #12461

Merged
merged 11 commits into from
Mar 22, 2024
Merged

Conversation

cedric-cordenier
Copy link
Contributor

No description provided.

* Add hardcoded YAML workflow and parsing

* Execute an individual step

* Execute an individual step
Copy link
Contributor

I see that you haven't updated any README files. Would it make sense to do so?

@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from bbdac51 to e5f277e Compare March 18, 2024 16:02
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from e5f277e to 3fb38d8 Compare March 18, 2024 16:14
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from 3fb38d8 to da4b39b Compare March 18, 2024 16:54
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from da4b39b to f1f16f0 Compare March 18, 2024 17:06
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from f1f16f0 to d0b2137 Compare March 18, 2024 17:19
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from d0b2137 to 8d8ae47 Compare March 18, 2024 17:40
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from 8d8ae47 to e90f8a8 Compare March 19, 2024 10:48
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from e90f8a8 to 6a33827 Compare March 19, 2024 11:39
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from 6a33827 to 539e083 Compare March 19, 2024 12:08
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from 539e083 to d17b0c2 Compare March 19, 2024 13:39
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from d17b0c2 to 4c2a220 Compare March 19, 2024 14:22
@cedric-cordenier cedric-cordenier force-pushed the ks-87-workflow-engine-scheduler branch from 4c2a220 to b7e188b Compare March 19, 2024 14:35
newWorkerTimeout time.Duration

// Used for testing to wait for an execution to complete
xxxExecutionFinished chan string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge fan of these kind of hooks. Can we maybe use a mock target in tests and wait for it to be called?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same... I initially used a mock target but that leads to a race since the engine will do some processing (like updating internal state) after the target has been called. It also won't notify you if the workflow has errored for some reason.

@cedric-cordenier cedric-cordenier added this pull request to the merge queue Mar 22, 2024
@bolekk bolekk removed this pull request from the merge queue due to a manual request Mar 22, 2024
Copy link
Contributor

@HenryNguyen5 HenryNguyen5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

going to let someone more experienced like @bolekk to approve, LGTM though!

// 2. Registers each step's capability to this workflow
// 3. Registers for trigger events now that all capabilities are resolved
//
// Steps 1 and 2 are retried every 5 seconds until successful.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want a retry limit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... probably not -- I think this should be a temporary thing, and if we're not able to initialize the workflow we should keep trying since the NOP has chosen to run the job.

return nil
}

// If the capability is already cached, that means we've already registered it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// If the capability is already cached, that means we've already registered it
// If the capability instance already exists, that means we've already registered it

@cedric-cordenier cedric-cordenier added this pull request to the merge queue Mar 22, 2024
Merged via the queue into develop with commit 4fe2e77 Mar 22, 2024
137 of 138 checks passed
@cedric-cordenier cedric-cordenier deleted the ks-87-workflow-engine-scheduler branch March 22, 2024 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants