airflow.providers.amazon.aws.triggers.eks

Module Contents

Classes

EksCreateClusterTrigger

Trigger for EksCreateClusterOperator.

EksDeleteClusterTrigger

Trigger for EksDeleteClusterOperator.

EksCreateFargateProfileTrigger

Asynchronously wait for the fargate profile to be created.

EksDeleteFargateProfileTrigger

Asynchronously wait for the fargate profile to be deleted.

EksCreateNodegroupTrigger

Trigger for EksCreateNodegroupOperator.

EksDeleteNodegroupTrigger

Trigger for EksDeleteNodegroupOperator.
class airflow.providers.amazon.aws.triggers.eks.EksCreateClusterTrigger(cluster_name, waiter_delay, waiter_max_attempts, aws_conn_id, region_name=None)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Trigger for EksCreateClusterOperator.

The trigger will asynchronously wait for the cluster to be created.

Parameters

  • cluster_name (str) – The name of the EKS cluster

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

  • region_name (str | None) – Which AWS region the connection should use. If this is None or empty then the default boto3 behaviour is used.

hook()[source]

Override in subclasses to return the right hook.

async run()[source]

Run the trigger in an asynchronous context.

The trigger should yield an Event whenever it wants to fire off an event, and return None if it is finished. Single-event triggers should thus yield and then immediately return.

If it yields, it is likely that it will be resumed very quickly, but it may not be (e.g. if the workload is being moved to another triggerer process, or a multi-event trigger was being used for a single-event task defer).

In either case, Trigger classes should assume they will be persisted, and then rely on cleanup() being called when they are no longer needed.

class airflow.providers.amazon.aws.triggers.eks.EksDeleteClusterTrigger(cluster_name, waiter_delay, waiter_max_attempts, aws_conn_id, region_name, force_delete_compute)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Trigger for EksDeleteClusterOperator.

The trigger will asynchronously wait for the cluster to be deleted. If there are any nodegroups or fargate profiles associated with the cluster, they will be deleted before the cluster is deleted.

Parameters

  • cluster_name – The name of the EKS cluster

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

  • region_name (str | None) – Which AWS region the connection should use. If this is None or empty then the default boto3 behaviour is used.

  • force_delete_compute (bool) – If True, any nodegroups or fargate profiles associated with the cluster will be deleted before the cluster is deleted.

serialize()[source]

Return the information needed to reconstruct this Trigger.

Returns

Tuple of (class path, keyword arguments needed to re-instantiate).

Return type

tuple[str, dict[str, Any]]

hook()[source]

Override in subclasses to return the right hook.

async run()[source]

Run the trigger in an asynchronous context.

The trigger should yield an Event whenever it wants to fire off an event, and return None if it is finished. Single-event triggers should thus yield and then immediately return.

If it yields, it is likely that it will be resumed very quickly, but it may not be (e.g. if the workload is being moved to another triggerer process, or a multi-event trigger was being used for a single-event task defer).

In either case, Trigger classes should assume they will be persisted, and then rely on cleanup() being called when they are no longer needed.

async delete_any_nodegroups(client)[source]

Delete all EKS Nodegroups for a provided Amazon EKS Cluster.

All the EKS Nodegroups are deleted simultaneously. We wait for all Nodegroups to be deleted before returning.

async delete_any_fargate_profiles(client)[source]

Delete all EKS Fargate profiles for a provided Amazon EKS Cluster.

EKS Fargate profiles must be deleted one at a time, so we must wait for one to be deleted before sending the next delete command.

class airflow.providers.amazon.aws.triggers.eks.EksCreateFargateProfileTrigger(cluster_name, fargate_profile_name, waiter_delay, waiter_max_attempts, aws_conn_id, region_name=None)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Asynchronously wait for the fargate profile to be created.

Parameters

  • cluster_name (str) – The name of the EKS cluster

  • fargate_profile_name (str) – The name of the fargate profile

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

hook()[source]

Override in subclasses to return the right hook.

class airflow.providers.amazon.aws.triggers.eks.EksDeleteFargateProfileTrigger(cluster_name, fargate_profile_name, waiter_delay, waiter_max_attempts, aws_conn_id, region_name=None)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Asynchronously wait for the fargate profile to be deleted.

Parameters

  • cluster_name (str) – The name of the EKS cluster

  • fargate_profile_name (str) – The name of the fargate profile

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

hook()[source]

Override in subclasses to return the right hook.

class airflow.providers.amazon.aws.triggers.eks.EksCreateNodegroupTrigger(cluster_name, nodegroup_name, waiter_delay, waiter_max_attempts, aws_conn_id, region_name=None)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Trigger for EksCreateNodegroupOperator.

The trigger will asynchronously poll the boto3 API and wait for the nodegroup to be in the state specified by the waiter.

Parameters

  • cluster_name (str) – The name of the EKS cluster associated with the node group.

  • nodegroup_name (str) – The name of the nodegroup to check.

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

  • region_name (str | None) – Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

hook()[source]

Override in subclasses to return the right hook.

class airflow.providers.amazon.aws.triggers.eks.EksDeleteNodegroupTrigger(cluster_name, nodegroup_name, waiter_delay, waiter_max_attempts, aws_conn_id, region_name=None)[source]

Bases: airflow.providers.amazon.aws.triggers.base.AwsBaseWaiterTrigger

Trigger for EksDeleteNodegroupOperator.

The trigger will asynchronously poll the boto3 API and wait for the nodegroup to be in the state specified by the waiter.

Parameters

  • cluster_name (str) – The name of the EKS cluster associated with the node group.

  • nodegroup_name (str) – The name of the nodegroup to check.

  • waiter_delay (int) – The amount of time in seconds to wait between attempts.

  • waiter_max_attempts (int) – The maximum number of attempts to be made.

  • aws_conn_id (str | None) – The Airflow connection used for AWS credentials.

  • region_name (str | None) – Which AWS region the connection should use. (templated) If this is None or empty then the default boto3 behaviour is used.

hook()[source]

Override in subclasses to return the right hook.

Was this entry helpful?