airflow.providers.amazon.aws.triggers.batch

Module Contents

Classes

BatchJobTrigger

Checks for the status of a submitted job_id to AWS Batch until it reaches a failure or a success state.

BatchCreateComputeEnvironmentTrigger

Asynchronously poll the boto3 API and wait for the compute environment to be ready.

class airflow.providers.amazon.aws.triggers.batch.BatchJobTrigger(job_id, region_name=None, aws_conn_id='aws_default', waiter_delay=5, waiter_max_attempts=720)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Checks for the status of a submitted job_id to AWS Batch until it reaches a failure or a success state.

Parameters
  • job_id (str | None) – the job ID, to poll for job completion or not

  • region_name (str | None) – AWS region name to use Override the region_name in connection (if provided)

  • aws_conn_id (str | None) – connection id of AWS credentials / region name. If None, credential boto3 strategy will be used

  • waiter_delay (int) – polling period in seconds to check for the status of the job

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

hook()[source]

Override in subclasses to return the right hook.

class airflow.providers.amazon.aws.triggers.batch.BatchCreateComputeEnvironmentTrigger(compute_env_arn, waiter_delay=30, waiter_max_attempts=10, aws_conn_id='aws_default', region_name=None)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Asynchronously poll the boto3 API and wait for the compute environment to be ready.

Parameters
  • compute_env_arn (str) – The ARN of the compute env.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

  • region_name (str | None) – region name to use in AWS Hook

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

hook()[source]

Override in subclasses to return the right hook.

Was this entry helpful?