airflow.providers.amazon.aws.operators.kinesis_analytics
¶
Module Contents¶
Classes¶
Creates an AWS Managed Service for Apache Flink application. |
|
Starts an AWS Managed Service for Apache Flink application. |
|
Stop an AWS Managed Service for Apache Flink application. |
- class airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2CreateApplicationOperator(application_name, runtime_environment, service_execution_role, create_application_kwargs=None, application_description='Managed Service for Apache Flink application created from Airflow', **kwargs)[source]¶
Bases:
airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator
[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook
]Creates an AWS Managed Service for Apache Flink application.
See also
For more information on how to use this operator, take a look at the guide: Create an Amazon Managed Service for Apache Flink Application
- Parameters
application_name (str) – The name of application. (templated)
runtime_environment (str) – The runtime environment for the application. (templated)
service_execution_role (str) – The IAM role used by the application to access services. (templated)
create_application_kwargs (dict[str, Any] | None) – Create application extra properties. (templated)
application_description (str) – A summary description of the application. (templated)
aws_conn_id – The Airflow connection used for AWS credentials. If this is
None
or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).region_name – AWS region_name. If not specified then the default boto3 behaviour is used.
verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
- class airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2StartApplicationOperator(application_name, run_configuration=None, wait_for_completion=True, waiter_delay=60, waiter_max_attempts=20, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]¶
Bases:
airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator
[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook
]Starts an AWS Managed Service for Apache Flink application.
See also
For more information on how to use this operator, take a look at the guide: Start an Amazon Managed Service for Apache Flink Application
- Parameters
application_name (str) – The name of application. (templated)
run_configuration (dict[str, Any] | None) – Application properties to start Apache Flink Job. (templated)
wait_for_completion (bool) – Whether to wait for job to stop. (default: True)
waiter_delay (int) – Time in seconds to wait between status checks. (default: 60)
waiter_max_attempts (int) – Maximum number of attempts to check for job completion. (default: 20)
deferrable (bool) – If True, the operator will wait asynchronously for the job to stop. This implies waiting for completion. This mode requires aiobotocore module to be installed. (default: False)
aws_conn_id – The Airflow connection used for AWS credentials. If this is
None
or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).region_name – AWS region_name. If not specified then the default boto3 behaviour is used.
verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
- class airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2StopApplicationOperator(application_name, force=False, wait_for_completion=True, waiter_delay=60, waiter_max_attempts=20, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), **kwargs)[source]¶
Bases:
airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator
[airflow.providers.amazon.aws.hooks.kinesis_analytics.KinesisAnalyticsV2Hook
]Stop an AWS Managed Service for Apache Flink application.
See also
For more information on how to use this operator, take a look at the guide: Stop an Amazon Managed Service for Apache Flink Application
- Parameters
application_name (str) – The name of your application. (templated)
force (bool) – Set to true to force the application to stop. If you set Force to true, Managed Service for Apache Flink stops the application without taking a snapshot. (templated)
wait_for_completion (bool) – Whether to wait for job to stop. (default: True)
waiter_delay (int) – Time in seconds to wait between status checks. (default: 60)
waiter_max_attempts (int) – Maximum number of attempts to check for job completion. (default: 20)
deferrable (bool) – If True, the operator will wait asynchronously for the job to stop. This implies waiting for completion. This mode requires aiobotocore module to be installed. (default: False)
aws_conn_id – The Airflow connection used for AWS credentials. If this is
None
or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).region_name – AWS region_name. If not specified then the default boto3 behaviour is used.
verify – Whether to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html