Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: support Prefect Workers #41

Merged
merged 4 commits into from
Nov 21, 2024
Merged

feat: support Prefect Workers #41

merged 4 commits into from
Nov 21, 2024
+150 −26

Conversation

rphamle
Copy link
Collaborator

@rphamle rphamle commented Nov 20, 2024 •
edited
Loading

Description

This PR will enable Cascade to be used with Prefect Workers (and remain backwards compatibility with Prefect Agents).

Tests

Successfully tested a Flow Run in Prefect, using the cascade_test project of this branch (deployed with sq forge deploy dev)

@rphamle rphamle force-pushed the rphamle/MLT-833/support-workers branch 6 times, most recently from 8c50e13 to 2a387a0 Compare November 20, 2024 16:46
@rphamle rphamle force-pushed the rphamle/MLT-833/support-workers branch from 2a387a0 to d90ee62 Compare November 20, 2024 16:47
@rphamle rphamle marked this pull request as ready for review November 20, 2024 16:52
@damienrj damienrj requested a review from a team November 20, 2024 17:01
jhamet93
jhamet93 approved these changes Nov 20, 2024
View reviewed changes
Copy link
Collaborator

@jhamet93 jhamet93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome. Was this tested in a Prefect Flow to confirm ?

Upon merging, will need to bump the internal Cascade repo to pin to this new version.

@rphamle
Copy link
Collaborator Author

rphamle commented Nov 20, 2024

Looks awesome. Was this tested in a Prefect Flow to confirm ?

Upon merging, will need to bump the internal Cascade repo to pin to this new version.

How do I make a dev version of this package to test with?

@jhamet93
Copy link
Collaborator

Looks awesome. Was this tested in a Prefect Flow to confirm ?
Upon merging, will need to bump the internal Cascade repo to pin to this new version.

How do I make a dev version of this package to test with?

Image builder would be great for this. You could use poetry add <PATH_TO_CASCADE> --editable in the flow project which would link to this version of Cascade. Image Builder would then include this as part of the image build.

@rphamle rphamle force-pushed the rphamle/MLT-833/support-workers branch from e55ecb1 to 5bccb08 Compare November 21, 2024 21:59
@rphamle rphamle force-pushed the rphamle/MLT-833/support-workers branch from 5bccb08 to ba784be Compare November 21, 2024 22:13
@rphamle
Copy link
Collaborator Author

rphamle commented Nov 21, 2024

Looks awesome. Was this tested in a Prefect Flow to confirm ?
Upon merging, will need to bump the internal Cascade repo to pin to this new version.

How do I make a dev version of this package to test with?

Image builder would be great for this. You could use poetry add <PATH_TO_CASCADE> --editable in the flow project which would link to this version of Cascade. Image Builder would then include this as part of the image build.

Had to add a few things to this PR but successfully tested a Flow Run (details in PR description)

@rphamle rphamle merged commit 6ddbe75 into main Nov 21, 2024
4 checks passed
@rphamle rphamle deleted the rphamle/MLT-833/support-workers branch November 21, 2024 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jhamet93 jhamet93 jhamet93 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rphamle @jhamet93