airflow.providers.singularity.operators.singularity

Module Contents

Classes

SingularityOperator

Execute a command inside a Singularity container.
class airflow.providers.singularity.operators.singularity.SingularityOperator(*, image, command, start_command=None, environment=None, pull_folder=None, working_dir=None, force_pull=False, volumes=None, options=None, auto_remove=False, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a command inside a Singularity container.

Singularity has more seamless connection to the host than Docker, so no special binds are needed to ensure binding content in the user $HOME and temporary directories. If the user needs custom binds, this can be done with –volumes

Parameters

  • image (str) – Singularity image or URI from which to create the container.

  • auto_remove (bool | None) – Delete the container when the process exits. The default is False.

  • command (str | ast.AST) – Command to be run in the container. (templated)

  • start_command (str | list[str] | None) – Start command to pass to the container instance.

  • environment (dict[str, Any] | None) – Environment variables to set in the container. (templated)

  • working_dir (str | None) – Set a working directory for the instance.

  • force_pull (bool | None) – Pull the image on every run. Default is False.

  • volumes (list[str] | None) – List of volumes to mount into the container, e.g. ['/host/path:/container/path', '/host/path2:/container/path2'].

  • options (list[str] | None) – Other flags (list) to provide to the instance start.

  • working_dir – Working directory to set on the container (equivalent to the -w switch the docker client).

template_fields: Sequence[str] = ('command', 'environment')[source]
template_ext: Sequence[str] = ('.sh', '.bash')[source]
template_fields_renderers[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

on_kill()[source]

Override this method to clean up subprocesses when a task instance gets killed.

Any use of the threading, subprocess or multiprocessing module within an operator needs to be cleaned up, or it will leave ghost processes behind.

Was this entry helpful?