Running commands with Maestro

  • You execute maestro run with a YAML file including information about your run.
  • Your run includes a description and at least one study (a step in your run).
  • Your maestro run creates a directory with subdirectories and outputs for each study.
  • Check the status of a run via maestro status --disable-theme <directory>

Running Maestro on the cluster

  • You can run on the cluster by including info for Slurm in your Maestro YAML file
  • Maestro generates and submits its own batch scripts to your scheduler.
  • Steps without Slurm parameters will run on the login node by default.

MPI applications and Maestro

  • “Adding $(LAUNCHER) before commands signals to Maestro to use MPI via srun.”
  • “New Maestro runs can be grouped within a new directory specified by the environment variable OUTPUT_PATH”
  • You can use --dry to verify that the expected directory structure and scripts are created by a given Maestro YAML file.

Placeholders

  • Environment variables are placeholders defined under the env section of a Maestro YAML.
  • Parameters defined under global.parameters require lists of values and labels.

Chaining rules

  • We can control the order of steps run in a study by creating dependencies.
  • You can create a dependency with the depends: [{step name}] syntax.
  • Dependency syntax changes to depends: [{step name}_*] for parameterized steps.

Multiple parameters

  • “Multiple parameters can be defined under global.parameters.”
  • “Lists of values for all global parameters must have the same length; the Nth entries in the lists of values for all global parameters are used in a single job.”