Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planemo uploads to the Tool Shed include unwanted files #554

Open
gregvonkuster opened this issue Sep 2, 2016 · 4 comments · May be fixed by #1106
Open

planemo uploads to the Tool Shed include unwanted files #554

gregvonkuster opened this issue Sep 2, 2016 · 4 comments · May be fixed by #1106

Comments

@gregvonkuster
Copy link
Contributor

It seems planemo is uploading unwanted files to the Tool Shed with individual tool uploads. In some cases it uploads multiple tools from a demultiplexed git repository into a single tool repository, while in other cases it uploads unwanted files from other tools.

The individual ImageJ2 tools are demultiplexed in here: https://github.com/bgruening/galaxytools/tree/master/tools/image_processing/imagej2
Planemo uploaded each of these tools to the TTS, including every ImageJ2 tool in each individual repo:

Repository 'imagej2_adjust_threshold_binary' revision c05d862ac753 (repository tip)
Contents:
imagej2_adjust_threshold_binary.py
imagej2_adjust_threshold_binary.xml
imagej2_adjust_threshold_binary_jython_script.py
imagej2_analyze_particles_binary.py
imagej2_analyze_particles_binary_jython_script.py
imagej2_analyze_skeleton.py
imagej2_analyze_skeleton_jython_script.py
...

On the MTS, the meme tool repo contains the fimo tool's Python script:

Repository 'meme_meme' revision 487ce3fa1822 (repository tip)
Contents:
all_fasta.loc.sample
fimo_wrapper.py
meme.xml
test-data/
tool_data_table_conf.xml.sample
tool_dependencies.xml

There are other similar examples on both the TTS and MTS.
@jmchilton
Copy link
Member

I'm pretty much okay with this as default behavior. Despite my desire to do something about this for years - tools still don't have a way to specify tool files they require - so planemo just grabs pretty much everything that isn't a tool by default since these files don't hurt anything.

I consider the tool shed tarballs pretty much a build artifact and a black box - so there is no need in my mind to customize them extensively as long as they work. I understand that you and say @peterjc may care more about the tarball though - so it is possible to include and exclude files via directives in .shed.yml.

A future version of planemo and tools - may require that newer tools (using tool profile versions) require the tools annotate the files they need (this would be wonderful for pulsar) - and the default de-multiplexing behavior may change at that time (but probably only for these newer tools).

@peterjc
Copy link
Contributor

peterjc commented Sep 2, 2016

Yeah, I went for explicitly including the files via the .shed.yml manifest as I felt the default was too much magic.

@martenson
Copy link
Member

martenson commented Jun 7, 2017 •
edited
Loading

Duplicating the whole test-data folder for each repository for mothur enlarges the suite installation size from ~10MB to ~600MB (for ~120 tools).

Also it makes the repositories themselves more confusing and less standalone - unless you know the planemo magic behind, which shouldn't be the requirement.

so 👍

@bernt-matthias bernt-matthias linked a pull request Dec 6, 2020 that will close this issue
Open
3 tasks
@bernt-matthias
Copy link
Contributor

May one of you have a look at #1106?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

consider exclude in shed update bernt-matthias/planemo
5 participants
@peterjc @jmchilton @martenson @gregvonkuster @bernt-matthias