Skip to content

Workflow editor tutorial 1

Andrea Szöllössi edited this page Sep 15, 2017 · 24 revisions

Creating and executing a workflow on the Platform

Time to complete

10 minutes plus Platform execution time (approximately 3 minutes).

Objective

In this tutorial, we will:

  • view the apps available on the Platform
  • learn about the main features of the Rabix workflow editor
  • use the Rabix workflow editor create a simple workflow on the Platform containing a single tool and its inputs and outputs
  • run the workflow as a task on the Platform

The tool we will use in the workflow is FASTQC, which reads a set of sequence files and produces a quality report for each one as a zip file.

Note: In this tutorial, we use Rabix Composer to create a simple workflow directly on the Platform. This is a straightforward example to get you familiar with Rabix Composer. For a more complex workflow, you would often choose to develop and test the workflow locally, before pushing it to the Platform and testing it there. Workflow editor tutorial 2 shows you how to do that.

Prerequisites

Before starting this tutorial, you need to:

Step 1: View the apps available on the Platform

Open Rabix Composer. In the navigation pane, click Public Apps to see a list of all the public apps available on the Platform. You can organize the apps by toolkit or by category, or you can search for an app by name or partial name. Apps may be tools (shown as Tool icon) or workflows (shown as Workflow icon).

In the left-hand navigation pane, click My Apps to see your local files and your projects on the Platform. In the Local Files section you will see all the files you have in your local workspace, not just the tools and workflows. In the Platform section you will see all the projects you can access on the Platform. Click on a project to see the apps available in that project.

When you use the workflow editor to create workflows, you will be able to drag apps from any of these locations directly onto your workflow canvas.

Step 2: Use Rabix Composer to create a workflow on the Platform

Click + at the top right to create a new tab in the editor.

In the tab that is created, click Workflow, then Seven Bridges.

Specify Tutorial1 for the app name, and take care to set CWL Version to sbg:draft-2 (unlike Rabix Composer, the Platform does not yet fully support CWL 1.0). Leave the app type as Workflow, and choose a destination project from the drop-down.

Click Create to create the workflow on the Platform. The new workflow opens in the Graph view.

Step 3: Use the visual editor to add the FastQC tool to the workflow

In the navigation pane, click Public Apps and enter FastQC in the search box. Select the tool called FastQC and drag it from the left-hand side to the workflow canvas (tools have this icon Tool icon). The FastQC tool is placed on the workflow canvas with one input port and one output port.

Hover over the tool to see the name of each port.

Step 4: Add an input and an output to the workflow

Inputs and outputs allow us to specify the workflow input and output when the workflow is run as a task, or used as a component in a larger workflow.

Hover over the tool input port (labelled Input file) and drag it to the left until you see a dotted circle. Release it to add a workflow input labelled input_fastq. Repeat the action for the tool output port (labelled Report Zip) to add a workflow output report_zip.

Tip: If you want to delete a tool, connection, input or output from the workflow canvas, click it, then press the Backspace key on your keyboard, or click the dustbin icon Delete icon at the bottom right of the workflow editor. You can undo the deletion (or any other action) by selecting Edit > Undo (or Cmd+Z) from the main menu.

Step 5: Inspect the CWL code for the workflow

In the workflow editor, click Code to see the CWL for this workflow. It is possible to edit the CWL directly, then switch seamlessly back to one of the other views and view the changes. We won’t do that in this tutorial, so click Graph View to return to the workflow canvas.

Step 6: Save the workflow

Click the Save icon Save icon at the top right of the workflow canvas, add a revision note, then click Publish to push the app to the Platform.

Step 7: View the workflow on the Platform

In the workflow editor, click the Open on Platform icon Open on Platform icon to open the app in the Platform workflow editor. You can also open your app directly on the Platform (you will find your app on the Apps tab of the project you created it in). Note that the Platform workflow editor shows additional input ports for FastQC, because these are optional ports, and by default, Rabix Composer only shows mandatory ports to avoid clutter.

Step 8: Get sample data files needed to test the workflow

To test the workflow on the Platform, we are going to use paired-end read files, as these are small so the workflow will run quickly. On the Platform, select Data > Public Reference files and in the search box, enter merged-tumor.converted. You should see two files, merged-tumor.converted.pe_1.fastq and merged-tumor.converted.pe_2.fastq. Paired-end read files are normally processed as a pair, so select both files then click Copy to, and specify the destination project containing the workflow you have just created.

Step 9: Run the workflow on the Platform

In the workflow editor, click the Open on Platform icon Open on Platform icon to open the app in the Platform workflow editor.

In the Platform workflow editor, click Run (top left of the workflow editor). This prepares a task to run the workflow, but, despite the name, doesn’t actually run it. A draft task is created, and shows an error for the #input_fastq parameter. This simply means you must supply a value for this parameter before you can run the task.

On the Platform, click Select Files, then select the two files you copied to the project just now, merged-tumor.converted.pe_1.fastq and merged-tumor.converted.pe_2.fastq. Click Save. You have now supplied two files to feed into the workflow’s input.

Click Run. The task should take around 3-5 minutes to run. You will receive an email when it completes.

Step 10: View the workflow results

When you receive the notification email, click the link in the email to view the results. You should see that the task was successful and that two output files _4_merged-tumor.converted.pe_1_fastqc.zip and _4_merged-tumor.converted.pe_2_fastqc.zip were created.

These files contain information about the quality of the paired end read input files. In workflow editor tutorial 2, we will create a workflow that also includes a step to convert these zip files into a format that can be viewed on the Platform.

Clone this wiki locally