Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Roadmap #1928

Closed
martenson opened this issue Mar 15, 2016 · 3 comments
Closed

The Roadmap #1928

martenson opened this issue Mar 15, 2016 · 3 comments
Milestone

Comments

@martenson
Copy link
Member

martenson commented Mar 15, 2016

Main Galaxy Roadmap

as described in issues.rst

The next major review of this roadmap will be done by the Galaxy Project after the 18.01 release.
The discussion under this issue is moderated to keep the roadmap clear, but please feel free to comment on any part of it.

Priorities

Main/usegalaxy.org

UI rewrite

  • Eliminate makos [New Roadmap]
  • Routing issues @guerler @dannon [New Roadmap]
  • Older grids [New Roadmap]

Dataset collections

Interactive environments

[Move to new Roadmap]

Workflows and Job Running

Data Libraries

Tool Shed / tool administration

Galaxy Hub and Training Materials

  • Navigation improvements @dannon [Killed]
  • Incorporate admin training materials @natefoo [WIP]
  • Dockerize/isolate admin activities @natefoo [Killed]

Metrics collection

Uploading large numbers of datasets

Cloud

  • Galaxy-on-the-Cloud merges with the GVL → 17.09 release
  • CloudLaunch
    • Move out of beta
    • Write/update docs [WIP]
  • CloudBridge support for Azure & GCE
  • CloudMan
    • Core framework; integration w/ CloudBridge
    • Ability to deploy K8S [New Roadmap]
    • Launchable in containerized mode via CloudLaunch [Cloud Board]
    • UI [Cloud Board]
  • Containerized Galaxy deployment [Cloud Board]
  • ObjectStore

OAuth



Guided tours

  • easier to share
  • Create initial set of tours for 101, rna-seq, etc**. @nekrut [WIP]

Histories

[New Roadmap]

main / performance

  • Main performance stats @dannon
    • Build reports
    • Analyze/fix

Tools

Main Maintenance/Operations

  • Provide UI for Tool panel ordering/etc. @martenson @davebx @jmchilton [New Roadmap Future]
    • Separate the presentation layer of the tool panel to an extra file?
  • Labels on tool sections (derive labels dynamically)
  • Automate Installation AND maintenance (updates, deprecations, labels) @afgane @martenson

New / enhanced tools suites

  • Metagenomics *** @blankenberg
  • RNA-seq **
  • Long reads (PB and ONT) -- assembly, polishing, hybrid, variant calling, iso-seq *** @jxtx @nekrut @jgoecks @blankenberg
  • Viral population analysis tools** @nekrut @jgoecks @blankenberg
  • Methylation analyses - this is available mostly
  • R tool integration * @blankenberg
  • R tool integration wiki/github instruction page with Galaxy + Best practices to handle bioconductor tool integration
  • Chromatin Conformation @msauria

Workflows and Job Running

  • Job Piping/streaming from tool to tool

TS/dependencies

  • Remove support for uploading repos with more than one tool * @martenson
  • Automatically increment tool versions and don't expose changeset revisions.*
  • Offer version management to the tool author.

Workflow hosting service


  • This set of priorities was developed and extensively discussed during 2016 Galaxy Team meeting (March 14-15, Williamsport, PA).
  • Post 16.04 Review occurred on 5/24/2016 via hangouts.
  • Post 16.07 Review occurred on 9/xx/2016 via hangouts.
  • Post 16.10 Review occurred on 12/9/2016 via hangouts.
  • Post 17.01 Review occurred on 4/20/2017 after SAB meeting.
  • Post 17.05 Review occurred on 6/xx/2017 at GCC.
  • Post 17.09 Review occurred on 11/9/2017 on zoom. And again on 12/7.
@jmchilton
Copy link
Member

jmchilton commented Oct 11, 2016

@nekrut Wanted me to summarize n Slack conversation that ran away with itself on this issue. If I mis-characterize anyone - feel free to just edit my comment on Github.

  • @jxtx "We scale or we die."
  • Broad agreement that mulled and Conda are still good and should continue to be developed and improved. Small disagreements perhaps in how good and how rocky things have been - and whether things have been oversold.
  • Broad agreement that CWL should be implemented within Galaxy - and broad concern about whether it can really replace Galaxy tools. People think it should be implemented regardless and that the spec needs to continue or re-focus on how to build great interfaces from these abstract tool and workflow descriptions.
    • Concern with CWL is that it cannot be used to make "great" GUIs.
    • @jxtx has the concern with CWL that templating broadly speaking is better than input binding (on this point @jmchilton disagrees).
  • Agreement that @jmchilton should prepare a cwltool + conda + mulled demo for Biological Data Science poster in October. @jmchilton should also present this work to GA4GH working group on containers.
  • With respect to Dockstore - the push in Galaxy should be to consume things from Dockstore less than using it to store Galaxy tools.
  • Some discussion on "what is Galaxy" and "what is an idealized tool" - probably too abstract to be actionable.
  • There is concern from @jxtx that Galaxy shouldn't do workflow scheduling - we do too much already and we need to find places to collaborate. @jmchilton ❤️s collaboration but worries the process of adapting to something like Mesos/Pegasus/Condor/toil would mean losing features and doing things sub-optimally. @jmchilton in short - "enhancing the core things we do is easier than replacing the core of what we do".
  • @jmchilton believe scaling to 10,000 datasets in a collection across large workflows is very do-able if we focus on it - going higher is possible too if we ignore the GUI concerns.
  • Some in the community have the perception that Galaxy can't scale - this worries everyone and @jmchilton made the case that it can and has scaled. No real discussion of the public relations problem - focus is more on continuing to build a technically great product that actually can scale.
  • We need to work on dealing with cluster and infrastructure failures if we aren't going to pursue another backend, we should find collaborators to help implement these things if it will help save time for our limited devs and admins.
  • @jxtx in short - GA4GH is building failure detection into the spec and learning from our mistakes in this regard.
  • Predicting resource usage is hard, people want to work on it though.

@jxtx
Copy link
Contributor

jxtx commented Oct 11, 2016

There is concern from @jxtx that Galaxy shouldn't do workflow scheduling

I don't want to make that strong a comment. I think we should evaluate whether we can benefit from adopting (and potentially improving) a scheduling / planning library that is shared with others. The usual tradeoffs apply, but I think we should evaluate options.

@nsoranzo
Copy link
Member

nsoranzo commented Feb 7, 2018

Now maintained as a project: https://github.com/galaxyproject/galaxy/projects/8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants