airflow.providers.amazon.aws.sensors.glue_crawler
¶
Module Contents¶
Classes¶
Waits for an AWS Glue crawler to reach any of the statuses below. |
- class airflow.providers.amazon.aws.sensors.glue_crawler.GlueCrawlerSensor(*, crawler_name, **kwargs)[source]¶
Bases:
airflow.providers.amazon.aws.sensors.base_aws.AwsBaseSensor
[airflow.providers.amazon.aws.hooks.glue_crawler.GlueCrawlerHook
]Waits for an AWS Glue crawler to reach any of the statuses below.
‘FAILED’, ‘CANCELLED’, ‘SUCCEEDED’
See also
For more information on how to use this sensor, take a look at the guide: Wait on an AWS Glue crawler state
- Parameters
crawler_name (str) – The AWS Glue crawler unique name
aws_conn_id – The Airflow connection used for AWS credentials. If this is
None
or empty then the default boto3 behaviour is used. If running Airflow in a distributed manner and aws_conn_id is None or empty, then default boto3 configuration would be used (and must be maintained on each worker node).region_name – AWS region_name. If not specified then the default boto3 behaviour is used.
verify – Whether or not to verify SSL certificates. See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
botocore_config – Configuration dictionary (key-values) for botocore client. See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html