Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow using tool data bundles as inputs to reference data select parameters #17435

Merged
merged 14 commits into from
Feb 19, 2024

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Feb 7, 2024

Enables galaxyproject/idc#36, and we could later allow users to generate (or just upload) bundles.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

lib/galaxy/model/__init__.py Outdated Show resolved Hide resolved
@mvdbeek mvdbeek force-pushed the bundle_as_input branch 6 times, most recently from 89e3e8e to 28d3da4 Compare February 8, 2024 19:51
@mvdbeek
Copy link
Member Author

mvdbeek commented Feb 9, 2024

So ... this works for what we need in a first step. I guess these are the question marks I still have in my head:

  • do we want a specific datatype for bundles that is distinct from data_manager_json ? I'm simply checking that the file size is different from the total size (which it is if there is bundle data), but this might be a little cleaner
  • we're persisting {"src": "hda", "id": 1} as the parameter value for the select parameter (instead of the value string). I think we could make that a little cleaner if we modeled this as a union between a select parameter and a data input parameter. We don't have such a concept now though, and maybe that's separate work ? It would let us specify mutually exclusive parameters eventually though, which could be cool. Maybe this would also formalize the map-over / not mapped over distinction ? idk
  • should we make bundles uploadable ? this would be a really nice way to provide test data with iwc workflows, we could rework tool tests in a way that makes the tests runnable against non-managed instances (i.e. real server where you can't just mess with reference data). In a sense this would even make the cvmfs data more accessible to folks who can't mount cvmfs, since you could just reference the datacache url ...

@mvdbeek mvdbeek marked this pull request as ready for review February 9, 2024 14:28
@mvdbeek mvdbeek requested a review from jmchilton February 9, 2024 14:28
@github-actions github-actions bot added this to the 24.0 milestone Feb 9, 2024
@mvdbeek mvdbeek requested a review from a team February 13, 2024 14:10
@jmchilton
Copy link
Member

Amazing - thanks so much! I guess it needs a rebase.

@mvdbeek mvdbeek merged commit 0d9abfe into galaxyproject:dev Feb 19, 2024
54 of 55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants