Skip to content

stepup.queue.api

StepUp Queue API functions to build workflows.

sbatch(workdir, *, ext='.sh', rc=None, inp=(), env=(), out=(), vol=(), optional=False, pool=None, block=False)

Submit a SLURM job script.

The following filename conventions are used in the given working directory:

  • slurmjob{ext} is the job script to be submitted.
  • slurmjob.log is StepUp Queue’s log file keeping track of the job’s status.
  • slurmjob.out is the job’s output file (written by SLURM).
  • slurmjob.err is the job’s error file (written by SLURM).
  • slurmjob.ret is the job’s return code (written by a wrapper script).

Hence, you can only have one job script per working directory, and it is strongly recommended to use meaningful directory names. Within the directory, try to use as much as possible exactly the same file names for all jobs.

When the step is executed, it will submit the job or skip this if it was done before. If submitted, the step will wait until the job is finished. If already finished, the step will essentially be a no-op.

See step() documentation in StepUp Core for all optional arguments. and the return value.

Parameters:

  • ext (str, default: '.sh' ) –

    The filename extension of the jobscript. The full name is f"slurmjob{ext}". Extensions .log, .out, .err and .ret are not allowed.

  • rc (str | None, default: None ) –

    A resource configuration to be executed before calling sbatch. This will be executed in the same shell, right before the sbatch command. For example, you can run module swap cluster/something or prepare other resources. If multiple instructions are needed, put them in a file, e.g. rc.sh and pass it here as source rc.sh. In this case, you usually also want to include rc.sh in the inp list.

Source code in stepup/queue/api.py 
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def sbatch(
    workdir: str,
    *,
    ext: str = ".sh",
    rc: str | None = None,
    inp: Collection[str] | str = (),
    env: Collection[str] | str = (),
    out: Collection[str] | str = (),
    vol: Collection[str] | str = (),
    optional: bool = False,
    pool: str | None = None,
    block: bool = False,
):
    """Submit a SLURM job script.

    The following filename conventions are used in the given working directory:

    - `slurmjob{ext}` is the job script to be submitted.
    - `slurmjob.log` is StepUp Queue's log file keeping track of the job's status.
    - `slurmjob.out` is the job's output file (written by SLURM).
    - `slurmjob.err` is the job's error file (written by SLURM).
    - `slurmjob.ret` is the job's return code (written by a wrapper script).

    Hence, you can only have one job script per working directory,
    and it is strongly recommended to use meaningful directory names.
    Within the directory, try to use as much as possible exactly the same file names for all jobs.

    When the step is executed, it will submit the job or skip this if it was done before.
    If submitted, the step will wait until the job is finished.
    If already finished, the step will essentially be a no-op.

    See `step()` documentation in StepUp Core for all optional arguments.
    and the return value.

    Parameters
    ----------
    ext
        The filename extension of the jobscript.
        The full name is `f"slurmjob{ext}"`.
        Extensions `.log`, `.out`, `.err` and `.ret` are not allowed.
    rc
        A resource configuration to be executed before calling sbatch.
        This will be executed in the same shell, right before the sbatch command.
        For example, you can run `module swap cluster/something`
        or prepare other resources.
        If multiple instructions are needed, put them in a file, e.g. `rc.sh`
        and pass it here as `source rc.sh`.
        In this case, you usually also want to include `rc.sh` in the `inp` list.
    """
    if ext == "":
        ext = ".sh"
    elif ext[0] != ".":
        ext = f".{ext}"
    if ext in [".log", ".out", ".err", ".ret"]:
        raise ValueError(f"Invalid extension {ext}. The extension must not be .log, .out or .err.")
    action = "sbatch"
    if ext != ".sh":
        action += f" {ext}"
    if rc is not None:
        action += f" --rc={shlex.quote(rc)}"
    return step(
        action,
        inp=[f"slurmjob{ext}", *string_to_list(inp)],
        env=env,
        out=["slurmjob.out", "slurmjob.err", "slurmjob.ret", *string_to_list(out)],
        vol=["slurmjob.log", *string_to_list(vol)],
        workdir=workdir,
        optional=optional,
        pool=pool,
        block=block,
    )