Skip to content

Static Glob

Explicitly declaring static files with the static() function from the previous tutorial becomes tedious when dealing with many static files. To simplify matters, StepUp supports “glob” patterns, i.e., wildcards such as * and ?.

The glob() function is similar to static() and supports globbing, including some non-standard glob techniques discussed in the following tutorials.

Here, only the basic usage of glob() is covered. In the following tutorial, the use of glob() in conditionals is discussed. See Static Named Glob and Static Deferred Glob for more advanced use cases.

Example

Example source files: docs/getting_started/static_glob/

Create a subdirectory src/ with two files: sub/foo.txt and sub/bar.txt. Also, create a plan.py file with the following contents:

#!/usr/bin/env python3
from stepup.core.api import copy, glob, mkdir, static

static("src/")
mkdir("dst/")
for path_src in glob("src/*.txt"):
    copy(path_src, "dst/")

Make the plan executable and run it non-interactively:

chmod +x plan.py
stepup -n 1

This should produce the following output:

  DIRECTOR │ Listening on /tmp/stepup-########/director (StepUp 2.0.4)
   STARTUP │ (Re)initialized boot script
  DIRECTOR │ Launched worker 0
     PHASE │ run
     START │ ./plan.py
   SUCCESS │ ./plan.py
     START │ mkdir -p dst/
   SUCCESS │ mkdir -p dst/
     START │ cp -aT src/bar.txt dst/bar.txt
   SUCCESS │ cp -aT src/bar.txt dst/bar.txt
     START │ cp -aT src/foo.txt dst/foo.txt
   SUCCESS │ cp -aT src/foo.txt dst/foo.txt
  DIRECTOR │ Trying to delete 0 outdated output(s).
  DIRECTOR │ Stopping workers.
  DIRECTOR │ See you!

Note that all files found by the glob() function are declared static in the workflow. Hence, they cannot be outputs of other steps. (This is not optional.)

Try the Following

  • Run StepUp again without making any changes. You will notice that the ./plan.py step is executed again despite not having changed it. When StepUp starts from scratch, it has to assume that new files could have been added (since the last run) and that match the glob pattern. Hence, a step calling the glob() function cannot be skipped. (This can be avoided when using StepUp interactively. More on that later.)

  • Add a file src/egg.txt and run StepUp again with the same arguments. You will notice that known steps for sub/foo.txt and sub/bar.txt are skipped. A new step is added for src/egg.txt.