airflow.providers.amazon.aws.sensors.kinesis_analytics
¶
Module Contents¶
Classes¶
General sensor behaviour for AWS Managed Service for Apache Flink. |
|
Waits for AWS Managed Service for Apache Flink application to start. |
|
Waits for AWS Managed Service for Apache Flink application to stop. |
- class airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2BaseSensor(application_name, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]¶
Bases:
airflow.providers.amazon.aws.sensors.base_aws.AwsBaseSensor
[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook
]General sensor behaviour for AWS Managed Service for Apache Flink.
- Subclasses must set the following fields:
INTERMEDIATE_STATES
FAILURE_STATES
SUCCESS_STATES
FAILURE_MESSAGE
SUCCESS_MESSAGE
- Parameters
- class airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2StartApplicationCompletedSensor(*, application_name, max_retries=75, poke_interval=120, **kwargs)[source]¶
Bases:
KinesisAnalyticsV2BaseSensor
Waits for AWS Managed Service for Apache Flink application to start.
See also
For more information on how to use this sensor, take a look at the guide: Wait for an Amazon Managed Service for Apache Flink Application to start
- Parameters
application_name (str) – Application name.
deferrable – If True, the sensor will operate in deferrable mode. This mode requires aiobotocore module to be installed. (default: False, but can be overridden in config file by setting default_deferrable to True)
poke_interval (int) – Polling period in seconds to check for the status of the job. (default: 120)
max_retries (int) – Number of times before returning the current state. (default: 75)
aws_conn_id – The Airflow connection used for AWS credentials. If this is
None
or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).region_name – AWS region_name. If not specified then the default boto3 behaviour is used.
verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
- class airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2StopApplicationCompletedSensor(*, application_name, max_retries=75, poke_interval=120, **kwargs)[source]¶
Bases:
KinesisAnalyticsV2BaseSensor
Waits for AWS Managed Service for Apache Flink application to stop.
See also
For more information on how to use this sensor, take a look at the guide: Wait for an Amazon Managed Service for Apache Flink Application to stop
- Parameters
application_name (str) – Application name.
deferrable – If True, the sensor will operate in deferrable mode. This mode requires aiobotocore module to be installed. (default: False, but can be overridden in config file by setting default_deferrable to True)
poke_interval (int) – Polling period in seconds to check for the status of the job. (default: 120)
max_retries (int) – Number of times before returning the current state. (default: 75)
aws_conn_id – The Airflow connection used for AWS credentials. If this is
None
or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).region_name – AWS region_name. If not specified then the default boto3 behaviour is used.
verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html