Merge branch 'hotfix/1.12.2'

dockstore · Jul 18, 2022 · 8daba68 · 8daba68
2 parents aca83f3 + 0560465
commit 8daba68
Show file tree

Hide file tree

Showing 38 changed files with 289 additions and 166 deletions.
diff --git a/docs/_attic/icons_with_text.png b/docs/_attic/icons_with_text.png
diff --git a/docs/_attic/icons_with_text_new.png b/docs/_attic/icons_with_text_new.png
diff --git a/docs/advanced-topics/best-practices/best-practices-secure-fair-workflows.rst b/docs/advanced-topics/best-practices/best-practices-secure-fair-workflows.rst
@@ -54,7 +54,7 @@ Image / Container Best Practices
 
 - Use Dockerfiles to describe and configure images:
 
-		- See `Best Practices from Docker <https://www.docker.com/blog/intro-guide-to-dockerfile-best-practices/>`_ and `10 Simple Rules for Writing Dockerfiles for Reproducible Analysis <https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008316>`_.
+		- See `Best Practices from Docker <https://docs.docker.com/develop/develop-images/dockerfile_best-practices/>`_ and `10 Simple Rules for Writing Dockerfiles for Reproducible Analysis <https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008316>`_.
 		- Use a source control site (as mentioned above in :ref:`version-control-best-practices`) for versioning Dockerfiles (see a `simple example for versioning your Docker Image <https://medium.com/better-programming/how-to-version-your-docker-images-1d5c577ebf54>`_). 
 
 - Keep images light:

diff --git a/docs/advanced-topics/docker-alternatives.rst b/docs/advanced-topics/docker-alternatives.rst
@@ -5,15 +5,17 @@ Docker Alternatives
 ===================
 
 In some situations using Docker may be impractical because it requires all users to have root access.
-Several alternatives have been developed to make it possible to run rootless containers, including
-`Singularity <https://sylabs.io/docs/>`_ and 
-`rootless Docker <https://www.docker.com/blog/experimenting-with-rootless-docker/>`_.
 While Dockstore uses Docker by default, if necessary it may be possible to run your workflows with one
 of these alternatives. Because the call to Docker or an alternative is made by the workflow runner, usually cwltool
 or Cromwell, and not Dockstore directly, the difficulty of configuring a Docker alternative depends on the workflow
 type. Some Dockstore entries will run seamlessly without Docker, and some may be entirely incompatible in a rootless
 environment.
 
+Rootless Docker
+---------------
+`Experiments <https://medium.com/@tonistiigi/experimenting-with-rootless-docker-416c9ad8c0d6#:~:text=Docker%20engine%20provides%20lots%20of,Linux%20you%20need%20privileged%20capabilities.>`__ with a version of Docker that does not require root permissions have been proposed and experimented with for a while now. In December 2020 with the release of Docker Engine v20.10, "rootless Docker" has officially become part of Docker itself. While this does open the door for some systems, it does require special setup and it has some limitations. You can read all about it on `Docker's own docs about Rootless Mode <https://docs.docker.com/engine/security/rootless/>`__.
+
+
 Singularity
 -----------
 

diff --git a/docs/advanced-topics/dockstore-cli/advanced-features.rst b/docs/advanced-topics/dockstore-cli/advanced-features.rst
@@ -332,13 +332,15 @@ with the `Cromwell <https://github.com/broadinstitute/cromwell>`__ version liste
 +-------------+-----------------------+
 |     1.12    |          57           |
 +-------------+-----------------------+
+|     1.13    |          77           |
++-------------+-----------------------+
 
 Additionally, you can override the Cromwell version in your
 ``~/.dockstore/config`` using for example:
 
 ::
 
-    cromwell-version = 58
+    cromwell-version = 76
 
 The Dockstore CLI will attempt to download the version of Cromwell JAR file you specify from the `Cromwell
 download area <https://github.com/broadinstitute/cromwell/releases/>`__ to

diff --git a/docs/advanced-topics/snapshot-and-doi.rst b/docs/advanced-topics/snapshot-and-doi.rst
@@ -1,5 +1,5 @@
-Creating Snapshots and Requesting DOIs
-=============================================
+Creating Snapshots & Requesting DOIs
+====================================
 
 Introduction
 ------------------

diff --git a/docs/assets/images/docs/add-tool-button.png b/docs/assets/images/docs/add-tool-button.png
diff --git a/docs/assets/images/docs/delete-tool-button.png b/docs/assets/images/docs/delete-tool-button.png
diff --git a/docs/assets/images/docs/delete-tool.png b/docs/assets/images/docs/delete-tool.png
diff --git a/docs/assets/images/docs/docker/dockerhub-python-descript.png b/docs/assets/images/docs/docker/dockerhub-python-descript.png
diff --git a/docs/assets/images/docs/docker/dockerhub-search-python.png b/docs/assets/images/docs/docker/dockerhub-search-python.png
diff --git a/docs/assets/images/docs/github-app-tool.png b/docs/assets/images/docs/github-app-tool.png
diff --git a/docs/assets/images/docs/github-apps-multiple-tools-with-name.png b/docs/assets/images/docs/github-apps-multiple-tools-with-name.png
diff --git a/docs/assets/images/docs/github-apps-multiple-tools.png b/docs/assets/images/docs/github-apps-multiple-tools.png
diff --git a/docs/assets/images/docs/manage-gh-app-installation.png b/docs/assets/images/docs/manage-gh-app-installation.png
diff --git a/docs/assets/images/docs/my-tools.png b/docs/assets/images/docs/my-tools.png
diff --git a/docs/assets/images/docs/old-and-new-tool.png b/docs/assets/images/docs/old-and-new-tool.png
diff --git a/docs/assets/images/docs/single-tool-to-migrate.png b/docs/assets/images/docs/single-tool-to-migrate.png
diff --git a/docs/assets/images/docs/unpublish-tool-button.png b/docs/assets/images/docs/unpublish-tool-button.png
diff --git a/docs/assets/images/square/anvil_square.png b/docs/assets/images/square/anvil_square.png
diff --git a/docs/assets/images/square/anvil_text.png b/docs/assets/images/square/anvil_text.png
diff --git a/docs/assets/images/square/cavatica_square.png b/docs/assets/images/square/cavatica_square.png
diff --git a/docs/assets/images/square/cavatica_text.png b/docs/assets/images/square/cavatica_text.png
diff --git a/docs/assets/images/square/cgc_square.png b/docs/assets/images/square/cgc_square.png
diff --git a/docs/assets/images/square/cgc_text.png b/docs/assets/images/square/cgc_text.png
diff --git a/docs/assets/images/square/dnanexus_square.png b/docs/assets/images/square/dnanexus_square.png
diff --git a/docs/assets/images/square/dnanexus_text.png b/docs/assets/images/square/dnanexus_text.png
diff --git a/docs/assets/images/square/galaxy_text.png b/docs/assets/images/square/galaxy_text.png
diff --git a/docs/assets/images/square/nextflow_alt.png b/docs/assets/images/square/nextflow_alt.png
diff --git a/docs/assets/images/square/nextflowtower_square.png b/docs/assets/images/square/nextflowtower_square.png
diff --git a/docs/assets/images/square/nextflowtower_text.png b/docs/assets/images/square/nextflowtower_text.png
diff --git a/docs/assets/images/square/terra_text.png b/docs/assets/images/square/terra_text.png
diff --git a/docs/dockstore-introduction.rst b/docs/dockstore-introduction.rst
@@ -1,138 +1,46 @@
 About Dockstore
 ===============
 
-The Dockstore concept is simple; provide a place where users can share
-tools encapsulated in Docker and described with the `Common Workflow
-Language <https://www.commonwl.org/>`__ (CWL) or
-`Workflow Description Language <https://openwdl.org/>`__ (WDL),
-workflow languages used by members of and APIs created by the
-`GA4GH <https://www.ga4gh.org>`__ `Cloud Work
-Stream <http://ga4gh.cloud/>`__. This enables scientists, for example,
-to share analytical tools in a way that makes them machine readable and
-runnable in a variety of environments. While the Dockstore is focused on
-serving researchers in the biosciences, the combination of Docker +
-CWL/WDL can be used by anyone to describe the tools and services in
-their `Docker images <https://docs.docker.com/get-started/overview/#docker-objects>`__ in a standardized, machine-readable way.
-
-Dockstore also attempts to work with new and alternative
-languages/standards such as `Nextflow <https://www.nextflow.io/>`__ as
-popular challengers to CWL and WDL. We also work on the `GA4GH Tool
-Registry <https://github.com/ga4gh/tool-registry-service-schemas>`__ standard as
-a way of sharing data with workflow platforms and partners. We are also
-working with the task execution, workflow execution, and data transfer
-standards developing at the GA4GH.
-
-Built with Docker and Git
--------------------------
-
-.. note:: See `Docker Overview <https://docs.docker.com/get-started/overview/>`__ for an excellent overview about Docker.
-
-Docker repositories, like `Docker Hub <https://hub.docker.com/>`__,
-`Quay.io <https://quay.io/>`__ and `GitLab <https://about.gitlab.com>`__, and
-source control repositories like `GitHub <https://github.com>`__,
-`Bitbucket <https://bitbucket.org/>`__, and
-`GitLab <https://about.gitlab.com>`__, provide much of the infrastructure we
-need. Docker repositories allow users to build, publish, and share both
-public and private Docker images. However, the services lack
-standardized ways of describing how to invoke tools contained within
-Docker containers. Workflow descriptor languages provide standardised
-ways to define the inputs, parameterizations, and outputs of tools in a
-controlled way. Together, these resources provide the necessary tools to
-share analytical tools in a highly portable way, a key concern for the
-scientific community.
-
-.. figure:: /assets/images/docs/Ways_to_get_into_Dockstore.png
-   :alt: Overview
-
-   Overview
-
-Strategies
-----------
-
-You can register your tools and workflows on Dockstore in three broad
-ways as depicted above.
-
-A) Following our
-   :doc:`tutorials <getting-started/getting-started>`,
-   you can save your descriptors on GitHub, build your Docker image
-   automatically on Quay.io, and have Dockstore reach out and index your
-   tools
-
-B) Dockstore can retrieve your workflow descriptors from GitHub and
-   other source control methods. You are responsible for ensuring that
-   your descriptors point at valid Docker images.
-
-C) You will be able to use our new hosted workflows service to store
-   tools and workflows directly on dockstore.org to quickly get started,
-   prototype your ideas, and share workflows with a limited audience.
-
-In all three cases, you will have an opportunity to clean-up and
-configure your work before publishing to the rest of the world to see.
-
-You can mix and match in a number of these approaches. For example, you
-can go beyond our simple tutorials and automated approach to manually
-register tools that point at locations like Docker Hub, Seven Bridges,
-and Amazon ECR. You can substitute WDL and Nextflow for the descriptor
-language for workflows. You might even be able to mix and match
-descriptor languages eventually!
-
-Over time, we find "skinny" Docker images, those with single tools
-installed in them, are more helpful for extending and building new
-workflows with. That being said, "fat" Docker containers, which include
-multiple tools and even full workflows with frameworks like
-`SeqWare <https://seqware.github.io/>`__ or
-`Galaxy <https://galaxyproject.org/>`__, can have their place as well.
-Projects like the ICGC `PanCancer Analysis of Whole
-Genomes <https://dcc.icgc.org/pcawg>`__ (PCAWG) made use of "fat" Docker
-containers that had complex workflows that fully encapsulated alignment
-and variant calling. The self-contained nature of these Docker
-containers allowed for mobility between a wide variety of environments
-and greatly simplified the setup of these pipelines across a wide
-variety of HPC and cloud environments. Either approach works for the
-Dockstore so long as you can describe the tool or workflow inside the
-Docker container as a CWL/WDL-defined tool (which you can for most
-things).
+The Dockstore concept is simple: provide a place where users can share tools encapsulated in Docker and described with the workflow languages. This enables scientists to share analytical tools in a way that makes them machine readable and runnable in a variety of environments. While Dockstore is focused on serving researchers in the biosciences, the combination of Docker and workflow languages can be used by anyone to describe the tools and services in their `Docker images <https://docs.docker.com/get-started/overview/#docker-objects>`__ in a standardized, machine-readable way. 
+
+By providing an environment where scientific tools can easily be created, maintained, browsed, and launched, Dockstore aligns itself with several trends supporting open science. But it is more than just an open ecosystem to facilitate open science -- we're involved in shaping what open science looks like. Dockstore works closely with the `GA4GH <https://www.ga4gh.org>`__, and accordingly, Dockstore supports the languages commonly used by GA4GH's `Cloud Work Stream <http://ga4gh.cloud/>`__ members and APIs: `Common Workflow Language <https://www.commonwl.org/>`__ (CWL), `Workflow Description Language <https://openwdl.org/>`__ (WDL), `Nextflow <https://www.nextflow.io/>`__ (NFL), and `Galaxy <https://github.com/galaxyproject/gxformat2>`. We also work on the `GA4GH Tool Registry <https://github.com/ga4gh/tool-registry-service-schemas>`__ standard as a way of sharing data with workflow platforms and partners, in addition contributing to the development of workflow execution and data transfer standards at the GA4GH. 
+
+Building off Docker and Git
+---------------------------
+
+There are existing repositories for Docker images, such as  `Docker Hub <https://hub.docker.com/>`__, `Quay.io <https://quay.io/>`__, and `GitLab <https://about.gitlab.com>`__ which allow users to build, publish, and share both public and private Docker images. There are also source control repositories like `GitHub <https://github.com>`__, `Bitbucket <https://bitbucket.org/>`__, and `GitLab <https://about.gitlab.com>`__, which are based on the `git <https://git-scm.com/>`__ approach to source control. This infrastructure is important, and indeed, Dockstore can link to those services for multiple tasks. However, the services on their own lack standardized ways of describing how to invoke tools contained within Docker containers. This is where Dockstore and descriptor languages step in, providing standardized ways to define the inputs, parameterizations, and outputs of tools in a controlled way. The Dockstore website and CLI additionally implement GA4GH standards and APIs to make the execution of workflows and tools, whether that be on your local computer or on the cloud, easier and more standardized. Together, these resources provide the necessary tools to share analytical tools in a highly portable and reproducible way, a key concern for the scientific community.
 
 Promoting Standards
 -------------------
 
-We hope Dockstore provides a reference implementation for tool sharing
-in the sciences. The Dockstore is essentially a living and evolving
-proof of concept designed as a starting point for two activities that we
-hope will result in community standards within the GA4GH:
+Dockstore provides a reference implementation for sharing computational analysis procedures in the sciences. It is essentially a living and evolving proof-of-concept designed as a starting point for three activities that we hope will result in community standards within the GA4GH: 
 
--  a best practices guide for describing tools in Docker containers with
-   CWL/WDL/Nextflow
--  a minimal web service standard for registering, searching and
-   describing CWL-annotated Docker containers that can be federated and
-   indexed by multiple websites similar to `Maven
-   Central <https://search.maven.org/>`__
+-  a best practices guide for describing tools in Docker containers with descriptor languages
+-  a web service standard for registering, searching, and
+   describing annotated Docker images that can be federated and
+   indexed by multiple websites
+-  a CLI tool to interact with both the Dockstore webservice and local workflow executors such as
+   Cromwell or cwltool
 
-We also implement certain utilities such as file provisioning plugins
-that support the GA4GH DOS (Data Object Service) standard or
-command-line launchers that present a common interface across CWL and
-WDL as almost a polyfill to demonstrate what we wish to use over time
-natively.
+We also implement certain utilities such as file provisioning plugins that support the GA4GH DRS (Data Repository Service) standard or command-line launchers that present a common interface across CWL and WDL as a living and evolving example of what we wish to use over time natively.
 
 Building a Community
 --------------------
 
-Several large projects in the Biosciences, specifically cancer
-sequencing projects such as PCAWG, PrecisionFDA, the Broad Institute,
-the University of California Santa Cruz, and Cancer IT at Sanger have
-registered between 10 and 60 workflows each in Dockstore as of August
-2018. We hope this work will aid the community and promote the
-registration of a large number of high-quality workflows in the system.
+Several large projects in the biosciences have registered workflows on Dockstore, including cancer sequencing projects such as PCAWG, PrecisionFDA, the Broad Institute, the University of California at Santa Cruz, and Cancer IT at Sanger. Dockstore is also a component of several NIH-funded cloud ecosystems including BioData Catalyst, the AnVIL Project, and eLwazi. We hope this work will aid the community and promote the registration of a large number of high-quality workflows in the system.
+
+You can browse workflows uploaded by such projects in `our organizations page <https://dockstore.org/organizations>`__.
 
 Future Plans
 ------------
 
-We plan on expanding the Dockstore in several ways over the coming
-months. Please see our `issues
-page <https://github.com/dockstore/dockstore/issues>`__ for details and
-discussions.
+Dockstore is continuously evolving, and as an open-source project, you can get involved too! Please see our `issues page <https://github.com/dockstore/dockstore/issues>`__ for details and discussions. For longer term plans, please see our `roadmap page <https://github.com/dockstore/dockstore/wiki/Dockstore-Roadmap>`__.
+
+
+Ready to get started?
+---------------------
 
-For longer term plans, please see our `roadmap page <https://github.com/dockstore/dockstore/wiki/Dockstore-Roadmap>`__.
+See our :doc:`Getting Started series </getting-started/getting-started>` for a walkthrough of creating your first workflow and uploading it to Dockstore!
 
 
 .. discourse::