This lesson is still being designed and assembled (Pre-Alpha version)

Introduction to Workflows with Common Workflow Language

For Contributors

Use the Episodes menu above to browse through the pages for indivual blocks of learning objectives. These pages can be used for designing challenges/exercises for the tutorial, as described in detail on this Issue. Note: the order of these episode pages is arbitrary and these pages may be removed/rearranged/merged later in the lesson development process.

You may find it helpful during lesson/exercise design to refer to these draft concept maps for the tutorial material

Example rendered exercise

This is the body of the challenge.

Solution

This is the body of the solution.

Prerequisites

This tutorial guides you through the the fundamentals of designing and building an analysis workflow. It assumes no previous knowledge or experience of workflows or Common Workflow Language (CWL), but does assume some experience with the Unix command line.

Before following this tutorial, you should be comfortable working in a Unix command line environment and familiar with fundamental commands (cd, mv, mkdir, etc), piping and redirection, and simple Bash scripting, such as might be gained from following the [Software Carpentry][swc] lesson, [The Unix Shell][swc-shell]. You might also have some experience with running tasks on a remote machine (by ssh connection) and in a cluster (high performance computing) environment.

If you have previously written a workflow description, in CWL or another langauge, you may want to look instead at the User Guide.

Target Audience

This tutorial is aimed at researchers and research software engineers who would like to begin automating their analyses in workflows. If you’re unsure whether this tutorial is a good fit for you check the prerequisites listed above. You may also find our learner profiles helpful. These are also a useful resource during the lesson design process.

Learning Objectives

Goals

Below is a list of draft learning objectives, (very) roughly broken down into chunks that feel like they belong together. These chunks could form the basis for splitting tutorial material into episodes. Most blocks include one objective in bold: this is intended to flag it up as a higher-level objective, encapsulating multiple smaller, more fine-grained, objectives that follow. The higher-level objectives may be suitable as learning objectives for the tutorial as a whole (i.e. they would be shown on the landing page, used to guide development and structuring of material, etc) while the “sub-objectives” would apply to specific sections (episodes) of the tutorial, and be used to guide the creation of exercises and content of those episodes.

The chunks get a bit less well-defined towards the end. See [this page][objective-notes] for the original notes these objectives are based on, created during the March 2020 lesson development sprint.

After following one of these tutorials, learners will be able to:


Objectives that require further discussion, if (and how) they should be included.

Schedule

Setup Download files required for the lesson
00:00 1. Introduction Key question (FIXME)
00:00 2. Citing the tools in your workflow Key question (FIXME)
00:00 3. Turning a shell script into a workflow Key question (FIXME)
00:00 4. CWL workflow descriptions Key question (FIXME)
00:00 5. Debugging workflows Key question (FIXME)
00:00 6. Documenting your workflow Key question (FIXME)
00:00 7. Workflows as dependency graphs Key question (FIXME)
00:00 8. Iterative workflow development Key question (FIXME)
00:00 9. Capturing output Key question (FIXME)
00:00 10. Describing requirements Key question (FIXME)
00:00 11. Scatter/Gather in a CWL Workflow How can I parallelize running of tools in a workflow?
00:05 12. Adding your own script to a step How to include and run a script in a step at runtime?
Which requirements need to be specified?
How to capture output of a script?
00:05 13. Sketches as a design tool Key question (FIXME)
00:05 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.