Summary and Setup

This is a new lesson built with The Carpentries Workbench.

Tutorial exercises and examples refer to two systems called Ruby and Pascal, which offer identical environments.

If you are working on a different cluster, you will need to install the prerequisites yourself.

Prerequisites

Setting up your environment

How you set up your environment with the prerequisites will depend on where you are running this tutorial.

Some Maestro commands will work locally but others won’t make sense unless you’re connected to a cluster with Slurm installed. Wherever you’re working, you’ll need to install the prerequisite Python packages listed above and download the python plotting script.

The recommended way to install these packages is inside a Python virtual environment. You create one using the venv module and specifying the directory in which your environment will be contained:

BASH 

mkdir ~/venvs
python3 -m venv ~/venvs/maestro

This should create the folder at ~/venvs/maestro, containing

BASH 

ls ~/venvs/maestro

OUTPUT 

bin/  include/  lib/  lib64  pyvenv.cfg

Now you can activate the environment. If your shell is bash,

BASH 

source ~/venvs/maestro/bin/activate

Otherwise, use the activate command appropriate for your shell. For example,

BASH 

echo $SHELL

OUTPUT 

/bin/tcsh

BASH 

source ~/venvs/maestro/bin/activate.csh

Your terminal prompt should now have a (maestro) prefix, indicating that the environment was successfully activated. This environment includes pip, a Python package manager. Use it to install the required libraries:

BASH 

pip install amdahl maestrowf yamllint

Specifying these three custom libraries should be sufficient: their dependencies will be pulled in by pip automagically!

Finally, copy the plot_terse_amdahl_results.py script into your venv’s bin directory and make it executable:

BASH 

cd ~/venvs/maestro/bin
wget https://github.com/xorJane/maestro-workflow-lesson/raw/main/episodes/files/plot_terse_amdahl_results.py
chmod +x plot_terse_amdahl_results.py

If you are working on Ruby, Maestro is installed to a Python virtual environment with binaries in /usr/global/docs/training/janeh/maestro_venv/bin. Instructions for how to use these binaries are contained in the lesson.

You will need the python script at /usr/global/docs/training/janeh/maestro_venv/plot_terse_amdahl_results.py or directly from GitHub.

If you are on LC and trying to set up a virtual environment on your own, follow the instructions for “If on your own cluster”, but add the flag --system-site-packages when creating a virtual environment:

BASH 

mkdir ~/venvs
python3 -m venv --system-site-packages ~/venvs/maestro

How to connect over SSH

Details

The application you use to connect to your cluster will depend on the type of local machine you’re working on.

We recommend using Git BASH via git for windows.

Use Terminal.app or similar.

Use Terminal or similar.