Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Declarative Testing for Workflow Behaviors #18542

Merged
merged 11 commits into from
Jul 30, 2024

Conversation

jmchilton
Copy link
Member

test_workflows.py has long grown unwieldy and many tests are just running workflows and checking outputs. I think these should be broken out and done in a declarative manner. I was also pondering #18541 and trying to think about how and where to describe the expected behavior and I didn't think just adding another test to the monster file was the right approach. I think there are limitations to what is being done with framework test tools (added in 569c5db) but they are better than a bunch of Python test cases I believe. In this PR, I have created a framework for running workflow framework tests to match framework tool tests. I've migrated a few workflow tests to the framework and I have implemented my desired test for #18541.

The syntax is not exactly what Planemo uses for workflow tests... but it is close and I think we can converge there over time. The biggest change is just input descriptions - the workflow tests grew up around automatically generated fixtures and Planemo is optimized for users specifying actual files. So the input specification is different - but they already different in the string-ified YAML throughout the test framework. I've implemented the elements syntax introduced by @mvdbeek for Planemo in galaxyproject/planemo#1417.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@jmchilton jmchilton force-pushed the workflow_tests branch 3 times, most recently from 1b4e592 to b19d8c6 Compare July 15, 2024 15:29
@jmchilton jmchilton marked this pull request as ready for review July 15, 2024 15:29
@github-actions github-actions bot added this to the 24.2 milestone Jul 15, 2024
@jmchilton jmchilton force-pushed the workflow_tests branch 3 times, most recently from daf961b to 9b36ac2 Compare July 26, 2024 14:54
Copy link
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That does look cleaner, thank you!

lib/galaxy_test/workflow/__init__.py Outdated Show resolved Hide resolved
@jmchilton jmchilton merged commit a3326a9 into galaxyproject:dev Jul 30, 2024
51 of 54 checks passed
@nsoranzo nsoranzo deleted the workflow_tests branch August 3, 2024 15:21
@jdavcs jdavcs added the highlight/dev Included in admin/dev release notes label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants