This lesson is in the early stages of development (Alpha version)

Reproducible computational environments using containers: Introduction to Singularity

This lesson provides an introduction to using the Singularity container platform. Singularity is particularly suited to running containers on infrastructure where users don’t have administrative privileges, for example shared infrastructure such as High Performance Computing (HPC) clusters.

This lesson will introduce Singularity from scratch showing you how to run a simple container and building up to creating your own containers and running parallel scientific workloads on HPC infrastructure.

Prerequisites

There are two core elements to this lesson - running containers and building containers. The prerequisites are slightly different for each and are explained below.

Running containers: (episodes 1-5 and 8)

  • Access to a local or remote platform with Singularity pre-installed and accessible to you as a user (i.e. no administrator/root access required).
    • If you are attending a taught version of this material, it is expected that the course organisers will provide access to a platform (e.g. an institutional HPC cluster) that you can use for these sections of the material.
  • The platform you will be using should also have MPI installed (required for episode 8).

Building containers: (episodes 6 and 7) Building containers requires access to a platform with an installation of Singularity on which you also have administrative access. If you run Linux and are comfortable with following the Singularity installation instructions, then installing Singularity directly on your system is an option. However, we strongly recommend using the Docker Singularity container for this section of the material. Details are provided on how to use the container in the relevant section of the lesson material. To support building containers, the prerequisite is therefore:

  • Access to a system with Docker installed on which you can run the Docker Singularity container.

    OR

  • Access to a local or remote Linux-based system on which you have administrator (root) access and can install the Singularity software.

Please note that the version of Singularity used in this part of the course is version 3.5.3 which was the latest stable release at the time of writing. If you are installing Singularity on your own system for use in the course, you are recommneded to install version 3.5.3.

Schedule

Setup Download files required for the lesson
Day 1 00:00 1. Singularity: Getting started What is Singularity and why might I want to use it?
00:50 2. The Singularity cache Why does Singularity use a local cache?
Where does Singularity store images?
01:00 3. Using Singularity containers to run commands How do I run different commands within a container?
How do I access an interactive shell within a container?
01:15 4. Files in Singularity containers How do I make data available in a Singularity container?
What data is made available by default in a Singularity container?
01:35 5. Using Docker images with Singularity How do I use Docker images with Singularity?
01:50 6. Preparing to build Singularity images What environment do I need to build a Singularity image and how do I set it up?
02:25 7. Building Singularity images How do I create my own Singularity images?
03:25 8. Running MPI parallel jobs using Singularity containers How do I set up and run an MPI job from a Singularity container?
04:35 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.