Instructor Notes
When to use a workflow system?
You may be asked by the learners when you would need to use a workflow system. The paper Workflow systems turn raw data into scientific knowledge has a view on this:
So, do you need a workflow system? Not every task requires one, and there is a learning curve. Scripting usually suffices for one-off tasks and when working out the pipeline itself. The tipping point, most agree, comes when you need to run the same workflow over and over again, or if the data are likely to be published.
Which workflow system to use?
The paper Using prototyping to choose a bioinformatics workflow management system discusses why they chose nextflow compared to other workflow systems. It also has a view about when to use workflow systems:
We conclude that many […] multistep data analysis workflows can be rewritten in a workflow management system, and we advocate prototyping as a low-cost (both time and effort) way of making an informed selection of software for use within a research project.
Nextflow echo option
It can be useful to use the -process.echo option to echo
output from a process when you run Nextflow .
nextflow run main.nf -process.echoNextflow video
The nextflow lesson material has been adapted from the sequera-labs training course link here there is video of this material being presented here
Code Editor
The preferred code editor is Visual Studio Code. Visual Studio Code has a syntax highlighting extension that adds the Nextflow language support.
Getting Help
For questions about the main Nextflow tool, use the Nextflow Gitter chat community: https://gitter.im/nextflow-io/nextflow
Many nf-core resources are to be found on this website. If in doubt, get in touch with the nf-core community via Slack.
The nf-core Slack channel has a #carpentries-course channel for asking questions about the course.
Nextflow REPL console
Nextflow has a console graphical interface. The Nextflow console is a REPL (read-eval-print loop) environment that allows one to quickly test part of a script or pieces of Nextflow code in an interactive manner.
It is a handy tool that allows one to evaluate fragments of Nextflow/Groovy code or fast prototype a complete pipeline script.
See the blog post here for more information.
Next Steps
Nextflow produces a blog post each year with links to training materials and workshops. This is a good website to show learners who want to learn more.
References
- Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nat Biotechnol. 2017;35(4):316-319. doi: 10.1038/nbt.3820). doi: 10.1038/nbt.3820.
- Ewels PA, Peltzer A, Fillinger S, et al. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020;38(3):276-278. DOI: 10.1038/s41587-020-0439-x
- Perkel JM. Workflow systems turn raw data into scientific knowledge. Nature. 2019;573(7772):149-150. DOI: 10.1038/d41586-019-02619-z
- Jackson M, Wallace EWJ, Kavoussanakis K, PLoS Computational Biology. Using prototyping to choose a bioinformatics workflow management system. 2021; 17 :e1008622. doi: 10.1371/journal.pcbi.1008622
Getting Started with Nextflow
Instructor Note
The nextflow scripts for each lesson episode are available in the
scripts directory created during the course setup. You
should copy the script into the current directory
For example, to copy the script for this lesson episode, run the following command:
Workflow parameterisation
Channels
Instructor Note
Create a nextflow script called to channel.nf for the
following examples. Explain that as you are just setting up channels you
do not need to add any processes or a workflow block.
Run using:
Processes
Instructor Note
The following section on python is meant to be run by the instructor not the learners. It is meant to be a demonstration of the different ways to run a process. This can be skipped for time.
Processes Part 2
Workflow
Operators
Reporting
Nextflow configuration
Instructor Note
This following exercise necessitates the installation of conda, a package and environment management software, on your system for downloading and installing the FASTP software. If conda is not installed, then this exercise should be omitted.